(SPEECH) [AUDIO LOGO] (DESCRIPTION) Logo: OTAN. -- A white figure reaches up and out of a blue circle. (SPEECH) - OTAN, Outreach and Technical Assistance Network. (DESCRIPTION) Text: Leading Adult Education Through Support for and the Effective Application of Technology. Professional Development. Teaching with Technology. OTAN Digests and Newsletters. News and Social Media. Annual Technology and Distance Learning Symposium. Online Resources and Video Presentations. Website, O.T.A.N. dot U.S. - Phone number, 916-228-2580. Slide: OTAN Online. Text: Technology and Distance Learning Symposium. California Adult Ed Technology and Distance Learning -- A Look at the Data. Presenters: Matthias Sturm, Adult Education Researcher. Anthony Burik, OTAN. March 2, 2024. Slide: Technology and Distance Learning Symposium 2024. March 1st to 2nd. Text: California Adult Ed Technology and Distance Learning -- A Look at the Data. Anthony Burik and Matthias Sturm. Technology and Distance Learning Symposium 2024. Saturday, March 2, 2024. a burik at O.T.A.N. dot U.S. and m sturm at S.F.U. dot C.A. Resources: bit dot L.Y. slash T.D.L. 24 folder. On the right is a QR code. Text: Hashtag T.D.L.S. 24. O.T.A.N. dot U.S. (SPEECH) ANTHONY BURIK: So why don't we get started officially? So Matthias want to introduce yourself? MATTHIAS STURM: Yes. Thank you, Anthony. I work with Anthony and previously with Penny Pearson, many of you know, on the Distance Learning Update Report. Now, if the TDLS changes the D to digital, will in the Distance Learning Update change. I'm wondering. AUDIENCE 1: It might. MATTHIAS STURM: OK, fair enough. You want to put it in a spot, that's cool. So I with every year with Anthony, and previously with Penny on that report. We use a lot of the data that CASAS collects from the agency that's in the WIOA funded agencies in California. And also we collect our own data through a survey that you might have seen early this year. And also the student technology intake survey is run by OTAN, right? ANTHONY BURIK: Yes MATTHIAS STURM: And is the teacher? No, that's not. ANTHONY BURIK: So there are two different surveys, the Teacher Tech Skills Survey and then the Student Tech Skills Survey. So we administer both of those, and monitor that for Adult Ed agencies. MATTHIAS STURM: So we use that data in that report. More and more, we comparing previous years to see if there are any trends, how that compares. Other than that, I'm from Canada work, I work for a not-for-profit out of Ottawa that works nationality in Canada called New Language Solutions that produces in the immigrant language sector that has a LMS called avenue.ca. So if you're interested, you can just look up avenue.ca. And the teacher training program, and I do a lot of evaluation and research there around the program. And Anthony, everybody knows. ANTHONY BURIK: So Anthony Burik, I'm one of the coordinators at coordinator for distance learning projects. So again, one of the things that I focus on in my work with OTAN is this report, this annual report that OTAN has been doing now for a number of years, Distance Learning and Technology in California of Adult Education. So that's been our focus now for the past few years. And again, our resources are here. They will be available after today, so if you want to come back and take a look at the slides, and the work that we do in the session today, it will be available. (DESCRIPTION) Slide: Agenda. Text: A little about OTAN. The Technology and Distance Learning Report. California Adult Education Digital Learning Guidance. A look at the data. Activity - I notice slash I wonder and discuss the data. Questions and final comments. Resources: bit dot L.Y. slash T.D.L. 24 folder. (SPEECH) So here's our agenda for our time together. So always good just to remind people about OTAN. Who's here for the first time ever at TDLS? Ben, I know of a couple of people in the back. Welcome. So just a reminder about OTAN, who we are. And then again, we'll talk a little bit about the report itself. We also want to make sure folks know about the Digital Learning Guidance. I know there was a session about that this morning. Some of you might have attended that session, but-- AUDIENCE 1: Perhaps-- ANTHONY BURIK: Christina was one of our presenters this morning, so she has more information to share with you all. But we want to talk about the report that we've been doing for a number of years, plus the guidance, which is relatively new, and how those two are starting to come together. So we'll talk about that. Matthias mentioned, we'll take a look at some of the data, and then we'll do some work together. Well, actually, you'll do some work independently for a few minutes, and then we'll come together and do some group work. And then again, as questions come up, if you have any questions, please ask us. And that's it. One thing, I'll just say is, so this is a hybrid session. (DESCRIPTION) Slide: California State Leadership Projects. Logos: California. CalPro. CASAS. OTAN. (SPEECH) So by now you probably have sat in on a couple of hybrid sessions. So let's just be mindful of our online folks. So if there's a lot of chatter in the room at the same time, they can't hear what's going on. So if we can just go one at a time, raise your hand if you need to, so we are respectful of the online folks. Just a reminder about OTAN, so in California, there are three state leadership projects for adult education, OTAN is one of them, we focus on technology training, but also CASAS is one of the projects, and also CALPRO is the third project. So we all work together to support adult education in California. So that's who we are. Everyone knows what OTAN is, right? Or is there anybody brand new to OTAN? AUDIENCE 1: He's new. ANTHONY BURIK: Brand new. AUDIENCE 1: He's our newest person. ANTHONY BURIK: OK well, welcome. AUDIENCE 1: Pablo Santos. ANTHONY BURIK: Hi Pablo. Welcome. MATTHIAS STURM: Welcome. ANTHONY BURIK: So, like I say, we are one of the state leadership projects supporting adult ed in California. So please come to our trainings, our webinars, tell your friends all that good stuff. (DESCRIPTION) Slide: OTAN, Outreach and Technical Assistance Network. Text: Follow Us. Twitter, slash O.T.A.N. LinkedIn, slash company slash O.T.A.N. underscore 2. Like Us, Facebook, slash O.T.A.N. Serves Adult Education. Subscribe, YouTube, slash O.T.A.N. Serves Adult Education. Text, Professional Development, News, Teach with Technology, Videos, TDLS, Online Resources. Website, O.T.A.N. dot U.S. -- Phone, 916-228-2580. (SPEECH) All the great things that we do here at OTAN. (DESCRIPTION) Slide: A look at the data. Text: Technology and Distance Learning California Update Background. T.D.L.P. since 2001 re. ongoing/proposed technology integration, teacher tech skills self-assessments, and learner tech use. Data from N.R.S., AEFLA Program Implementation Survey, C.I.P. teacher self-assessment, Student Technology Intake Survey. OTAN focus groups in P.Y. 2020 to 2021 to deepen understanding of remote instruction during and after the COVID-19 pandemic. OTAN survey with agencies this year, adding depth to the findings for P.Y. 2021 to 2022 and P.Y. 2022 to 2023 Program Implementation Survey. (SPEECH) So, Matthias, you want to get us going first on that report. MATTHIAS STURM: Sure. Don't want to spend too much time on it. There is some notes in there, a little bit more detail, if you're interested in the slide deck. So the Technology Distance Learning Update, it's been going on for quite some time. The difference, and you can see those previous ones on the report section on otan.us, it's always in the-- Has it always been in the Appendix F? It is so, hey. AUDIENCE 2: Mm-hmm. ANTHONY BURIK: Always for a while now, yeah. MATTHIAS STURM: And there's different sections in there over the different years, technology integration, teacher, tech skills, self-assessment, learner tech use. More recently, also stuff on digital equity, the way students access technology, what technology they have, if they have to share devices, things like that, things that have come up, especially have been of importance during the pandemic, or on the mind of many people during the pandemic. The data comes from the National Reporting System, the AEFLA Program Implementation Survey, the CIP, that's the continuous improvement-- No, Continuous Improvement-- ANTHONY BURIK: --Plan. MATTHIAS STURM: Thank you. --teacher self-assessment, and the Student Technology Intake Survey that OTAN's run a couple of years ago. So that produces a lot of quantitative data. And a couple of years ago, we've made a concerted effort to add more qualitative data to it, more of the voice of the field as well. And we first did that with focus groups, so speaking, more targetedly with programs, just after the pandemic, because there was lots to talk about. And we didn't feel like if we're just asking people in a survey that would come out kind of thing. Since then, the last couple of years, we've done it through a survey with a lot of open-ended questions, because also we wanted to cast the net wider, and not just zero in on just a few programs. So there's a lot of people who spend quite a bit of time giving us their feedback. And so that's the last point there. The OTAN survey, usually the beginning of January that adds quite a lot of depth. And the data that you see in every report is the program year. And now we'll be talking about the last program year, so '22 to '23, which ended the end of June, 2022. It takes a while for CASAS to get-- Usually CASAS gets the data out in November, something like that [INTERPOSING VOICES]-- ANTHONY BURIK: Usually the 3rd of December. MATTHIAS STURM: --when they start presenting on it, and then starting releasing it, so we can actually use it. And then we usually turn it over sometime in the spring. Then the report comes out and is publicly available. ANTHONY BURIK: Just on that point, so sometimes people say there's a bit of a lag in the data, because we're looking back at the '22, '23 year. And right now it's the start of 2024. So the time gap is widening. But one of the things that we're trying to do in this year's report is actually put in a couple of years of data, so not just '22, '23, but in some cases, if we can refer back to the previous year as well. So we're interested in trends, what the data is telling us in terms of maybe future directions, things like that. So that's a new feature for this year's report. MATTHIAS STURM: And what's then also important is, sometimes you get that through open-ended questions, but a lot in discussion with people, like what do those trends mean? You can see sometimes trends in the data, but it's really people on the ground who can maybe give us some idea of why is that actually happening, and what is actually the trend there. AUDIENCE 1: I wanted to say, because I work on the CIP as a region, and as my institution, so we can try to do it the best way we can. One of the things that is problematic with this survey, that might not have been three years ago, is Carolyn asked us to do it when the students enter. So that means, if I'm trying to create smart goals, I'm assessing students that entered in at this point in time. One thing, they're not the same students as this point in time. And two, I can't have any impact on what they come to me with. It almost seems like maybe we do this after 40 hours or something, I don't know. So are we're measuring apples or apples? That's the only thing I see, because I want to have real goals, I want to be able to show improvement. And that is the thing I'm struggling with. I don't know if it's OTAN I talk to, or the state, or CDE. But if we did it after a little time, then I could implement North Star, and then would have-- I don't know, that's right-- MATTHIAS STURM: That's a very good point, but just to make sure it's understood, we're not assessing, we just get the data aggregated for all programs. We're not looking at programs, specifically, or making any decisions, assumption, or whatever on any program evaluation, or anything like that, it's all aggregated together what we see. What CASAS sees, I don't know. How they deal with the data. I don't know. [INTERPOSING VOICES] AUDIENCE 1: --they're asking us to get this data, and make improvements on our program, which we want to do. MATTHIAS STURM: That's difficult, yeah. [INTERPOSING VOICES] And they want to have the data at a certain point in time, and that probably mandates more when they ask you for it, rather than what makes sense on the ground. ANTHONY BURIK: We try to be coldly objective with the reporting, but we realize that we want it also to be actionable and serviceable, and so it's important to point those challenges out. So let's move on here. (DESCRIPTION) A screenshot of a website with a report on the right and menu on the left. (SPEECH) So hopefully everybody, by now, is familiar with this document, the California Education Digital Learning Guidance. And so this came out officially in the fall of 2021, so it's been around for a couple of years. We have the guidance and some new resources hosted on the OTAN website. So one of the things about this guidance document is that it's really meant for California Adult Education agencies to think about all of the different components of adult education in a wide variety of topic areas. But it's really thinking about, create, in the field, how do we create really optimal digital learning experiences for our learners, no matter where the agency is in California. So again, this document has been around for a while, for a couple of years now. And last year it occurred to us that there are a lot of topics that are mentioned in the guidance document. Some of those topics are what we're collecting data on in these various locations, with the CASAS data or the OTAN data. So what are the connections between the topics that are listed in the guidance, and the data that might inform some of those topics? So we're still exploring, we're still thinking about it. We took a stab at some of these points last year on last year's report, and we're going to continue in that vein for this year. But with the guidance document itself, let me just check the Chat, sorry, for one second here. (DESCRIPTION) Slide: Digital Learning Guidance Chapters. Text: 1: Introduction. 2: Ensuring Equity and Access. 3: Foundations of Adult Education and Digital Learning. 4: Designing Flexible Learning Experiences. 5: Adopting Models that Work. 6: Data-Driven Instruction and Digital Assessments. 7: Fostering Healthy, Equitable, and Inclusive Digital Communities. (SPEECH) So Diana said, just make sure I don't repeat or summarize the questions. If I heard her, she should access her student survey results during the same year. Adel, do you want to just speak to that again, your point, about the-- AUDIENCE 1: So in the CIP, it's asking us to have a technology goal, but it also says create a technology, using the survey. So if I'm trying to do a SMART goal, and the SMART goal is for next year, based on the input from this year, but the next year, students I haven't provided any service to, because I'm giving it to them upon interest. I'm not sure how I'm supposed to make a measurable, specific goal for those students, because I can't give them a class before they become my student. ANTHONY BURIK: So again, that we present in the report, ultimately to the field, how useful is it going to be if we're thinking about either a reflection on what's really happening now, and then future planning, or thinking about particular SMART goals, or a particular technology goals. AUDIENCE 1: I get that. I can look at this, my students need this, I provide it. But how do I then measure that I provided that if I'm testing them upon entry rather than after [MUFFLED VOICE] ANTHONY BURIK: Got it. OK. So some things to think about. Thanks, Diana. So let's go back to the guidance for a second. So again, if you're not familiar with the guidance, it's broken down into an introduction. And then this is really the core of the guidance, these six chapters. So ensuring equity and access foundations of adult ed and digital learning, flexible learning experiences, models that work, data-driven instruction and assessments, and then healthy, equitable, and inclusive digital communities. And I bolded 2, 3, and 5 in particular, because when you dive into those three chapters, in chapter 2 of the guidance, we talk about devices, we talk about connectivity, we talk about digital literacy skills, which are certainly things that we're looking at in the various pieces of data that we're collecting. (DESCRIPTION) Text: Chapter 2: Ensuring Equity and Access. Chapter 3: Foundations of Adult Ed and Digital Learning. Chapter 5: Adopting Models that Work. (SPEECH) In the foundations chapter, which talks a lot about professional development, teacher professional development, again, that teacher skill tech skill survey is all about PD. And then in Chapter 5, Adopting Models that Work, we talk about distance ed, blended learning, and HyFlex, and other modalities, and we certainly collect data on that as well. (DESCRIPTION) Text: We attempt to explore these connections (and more) in the Tech and DL Report. (SPEECH) So again, we're looking at these chapters in particular as the intersection between the data that we're collecting, and specific topics that are mentioned, and can one inform the other, how can we learn about both of those things together with more study of the topics, as well as the data that might give us some insight into those particular topics. So before we continue by looking at the data, any questions so far about the data that we're collecting, the guidance? Any initial questions that folks have or online as well. (DESCRIPTION) Slide: Regular Classroom and Distance Learner Enrollments (1). P.Y. 2017 to 2023. Text: Figure 1 WIOA, Title II Adult Education Enrollments from P.Y. 2018 to 2019 to P.Y. 2022 2023 for Regular Classroom vs Distance Learner Enrollments Qualifying for N.R.S. Tables 4 and 4C (Source: CASAS 2023) Chart data table displayed in the following slide. On the right is a line graph titled Regular Classroom and Distance Learner Enrollments with a blue line for Regular Classroom and a red line for Distance Learning. The y-axis runs from 0 to 300,000 in increments of 100,000. The x-axis has the years 2018 to 19, 2019 to 20, 2020 to 21, 2021 to 22, and 2022 to 23. The blue line starts at 300,000, dips to about 120,000 in 2020 to 21, and grows to about 230,000 in 2022 to 23. The red line begins around 10,000, climbs to about 90,000 in 2020 to 21, then lowers to about 70,000 in 2022 to 23. (SPEECH) Great, OK. Matthias, you want to do some-- can you continue here? MATTHIAS STURM: Sure, so that's a multiyear graph there on regular a classroom enrollment and distance learning. Enrollment of-- the data comes from NRS tables 4 and 4c, which are publicly available if you Google them. And yeah, I guess it was really interesting how the regular classroom attendance obviously in 2020, 2021 was a lot lower. And the online learning, distance learning, online learning enrollments went up, and it's now diverging again post-pandemic. That kind of shows that. The next table on the next slide. (DESCRIPTION) Slide: Regular Classroom and Distance Learner Enrollments (2). A table lists the Regular Classroom enrollments for 2018 to 19 as 299,720, 2019 to 20 as 258, 201, 2020 to 21 as 134,492, 2021 to 22 as 187,371, and 2022 to 23 as 242, 068. The Distance Learning for the same years is 10,754, 70,463, 88,749, 67,588, and 65,466. Text: Figure 1. WIA slash WIOA Title II Adult Education Enrollments from 2018 to 2019 to 2022 to 2023 for Regular Classroom versus Distance Learner Enrollments Qualifying for N.R.S. Tables 4 and 4C. (Source: CASAS 2023). Another table below is titled, What is your feeling about learning online? Row one: I will continue to learn online. 2021 to 2022, Total, 17,449, 63.1%. 2022 to 2023, Total, 22,539 , 55.4%. Row two: I don't think I can learn online right now. 2021 to 2022, Total, 10,209, 36.9%. 2022 to 2023, 18,116, 44.6%. Text: Figure 10. Students' feelings about online learning. Student Technology Intake Survey Results from 2021 to 2022 to 2022 to 2023. (Source: OTAN 2022 and 2023.) (SPEECH) That's the same thing. So what's interesting here, at least to us, what we're talking about it earlier today and yesterday. What we've noticed is also the decline of distance learning, online learning participants between the last two program years that we have data on. The decline think from 2021 to 2022, after a lot of programs went back to in-person teaching in-- sorry, I know I got to get this right, September of '21. Yes, and then of course, the regular in-person classroom attendance went up again. So that's something we can discuss a little bit later, what you think what happened or what is happening at the moment in terms of where programs are going, in terms of offering online blended or in-person only. The service always ask, what is your-- this is in the student technology intake survey, what is your feeling about learning online? I think what's interesting here too, is that they can choose, I will continue to learn online in the between the last program year, the year before the last program year, 21/22, so it's declining. And whereas also we know anecdotally that a lot of students want to keep learning online, so that's also interesting too. And that might jive with what you're hearing from your learners or it may not at all. So we can talk about that too. ANTHONY BURIK: There's a quick question. MATTHIAS STURM: Yeah. AUDIENCE 1: I know our school has done a survey, but I don't remember how many languages we're talking in. MATTHIAS STURM: How many? AUDIENCE 1: Languages. MATTHIAS STURM: The students survey? AUDIENCE 1: Yes. MATTHIAS STURM: So it's offered in English. But the I believe on the survey form, there's a translator, a Google Translate tool. So I believe that agencies can make it available in other languages as well. AUDIENCE 1: Other languages as well, all right. MATTHIAS STURM: Yeah. What was it about? One year it was mandated, the intake. And now it's not anymore. -- ANTHONY BURIK: No, the student survey actually is-- MATTHIAS STURM: It still is? ANTHONY BURIK: It should be administered to all students. Yeah, there's no change from CDE, yeah. AUDIENCE 1: It was the-- MATTHIAS STURM: It's the teacher survey, which was optional. And it's still kind of optional, but we would really love for teachers to complete the survey. And actually at OTAN and CDE, we are in the process of-- yeah, we're in the process of revising that teacher tech skills survey. I'm not sure if it's going to be ready in the next program year or not. So wait and see on that. But yes. AUDIENCE 1: And the student survey is supposed to be done once a year or twice a year, or at the beginning of the semester or when is it? ANTHONY BURIK: Upon intake. Yeah so the MATTHIAS STURM: Yeah, and the student survey is available-- it's open all year. So whenever the students enter the agency. Yeah, or soon after. Well, this is the issue that Adele is raising is, after how many hours of instruction should we do it? But really it should be done at intake or soon after so-- yeah, AUDIENCE 1: Yeah, do we see our individual agencies results? MATTHIAS STURM: I don't know, but I would assume that not from us. AUDIENCE 1: Yeah. MATTHIAS STURM: We don't have it. All aggregated? We don't get that, but CASAS obviously has it so I don't know. So-- AUDIENCE 2: It's on OTAN website. If you have access to the portal for your WIOA You go under WIOA click on the box to the left and it will show you your aggregate data. It's hard to dig through. That's what we were discussing, but it is there. ANTHONY BURIK: Yeah, OK. Yeah, and you can always check with your CDE consultant on that. AUDIENCE 3: And if you're not WIOA, you can still get it by contacting support WIOA. MATTHIAS STURM: OK, thank you, Renee. ANTHONY BURIK: So OK, so just a couple comments again. A question about when to administer it. And then Darla said, we try to have students to do it during the first meeting, during an independent study, initial meeting, so the, I will continue to learn online is automatic, yes. MATTHIAS STURM: Ah, I think yes. ANTHONY BURIK: So we know-- Renee, Sorry. AUDIENCE 2: Well, I look at this question, and I wonder if students feel like they have a choice. MATTHIAS STURM: Yeah, I think you're right. AUDIENCE 2: So for example-- I'm sorry, can you go back? ANTHONY BURIK: Yeah. MATTHIAS STURM: Yeah. AUDIENCE 2: So I will continue to learn online. If they know that they're not going to be taking an online class and their only choice is to be in person, well, then they're probably going to say no. And then I don't think I can learn online right now. They probably don't think that they can learn or it maybe not an option. And so I wonder if we're creating barriers where we don't mean to be creating barriers. MATTHIAS STURM: It's a good point. It feels more almost like an exit or a benchmark question if you're-- ANTHONY BURIK: Yeah, so in our folder, we have a bunch of worksheets. So those kinds of comments are fantastic comments to add to the worksheets. So Renee keep that in mind. And Adele, keep your comment in mind too. So-- AUDIENCE 4: That's great. Yeah, absolutely. ANTHONY BURIK: Yeah, I'll just say, all of these surveys, they probably all can always be better. But it's helpful to hear from the field to-- what are the issues that you're coming up against? And to get that data, bring it back to CVE. Have those discussions about whether the surveys seem to be revised in any way. So like I said, the teachers-- we just revised the student tech skills survey recently, within the last couple of years. We're looking at the teacher skills survey right now. So, yeah. MATTHIAS STURM: Interesting. ANTHONY BURIK: OK, so let's move on to the next one. (DESCRIPTION) Slide: Digital Access and Equity (1). Text: How do you connect to the internet? A table has the columns, Wifi at home, Cell phone, Wifi in community, and Personal Hotspot. Wifi at home 2021 to 22, 76.8%, 2022 to 23, 72.8%. Cell phone 2021 to 22, 35.8%, 2022 to 23, 38.1%. Wifi in community, 2021 to 22, 4.4%, 2022 to 23, 6.0%. Personal Hotspot, 2021 to 22, 5.5%, 2022 to 23, 5.8%. Text: Figure 7. Students' ways to connect to the internet. Student Technology Intake Survey Results from 2021 to 2022 and 2022 to 2023. (Source: OTAN 2022 and 2023.) Below is another table titled, Which devices do you or can you use for online learning? with the columns Cell phone, Laptop or Computer, Tablet, and I don't have a device. Cell phone 2021 to 22, 64.1%, 2022 to 23, 64.4%. Laptop or Computer 2021 to 22, 64.3%. 2022 to 23, 59.7%. Tablet 2021 to 22, 19.7%, 2022 to 23, 18.6%. I don't have a device, 2021 to 22, 4.3%, 2022 to 23, 5.9%. Text: Figure 1. Students' use of devices for online learning. Student Technology Intake Survey Results from 2021 to 2022 and 2022 to 2023 (Source: OTAN 2022). (SPEECH) MATTHIAS STURM: So it also collects data on digital access and equity. Thanks, Anthony for fixing my typo there. It was not a wiki in community. It was Wi-Fi. ANTHONY BURIK: Wi-Fi, yes. MATTHIAS STURM: But wiki in community too, right-- Thanks. So yeah, here too again, we're comparing over the last two program years, Wi-Fi at home declined a little bit, which is also a little bit surprising since we always hear that people are getting better and better connected every time, every year. ANTHONY BURIK: And all this money is coming. All the federal money is coming for better connectivity, more access to devices. So this-- yeah. MATTHIAS STURM: Right, what's happening there, right? Cell phone use is up. OK, that's interesting. The Wi-Fi in the community, OK, so community-- there's more access to that. And personal hotspots. AUDIENCE 3: So I have a comment about that. I see a lot of money being-- I'm in a rural area, with a crap ton of granite. So I see a lot of money going through the areas-- like if you went to some of the areas of Apple Valley, they may be having access to three or four different devices that they can get from public entities, whether or not they have Wi-Fi. And the same problem of the digital divide is actually just making more of a divide, because in Luzerne, which is a tiny little community, they still have no Wi-Fi, except for the grocery store and the school. That has not changed since 2019, despite all the money that we put at it. If we're just buying things that are band-aids and we're not building more infrastructure-- which there has to be a reason for the companies to build those towers. And we're trying to say, hey, we're a really good place to shoot Beyonce films. Can someone help us out? [INTERPOSING VOICES] And that is what I see as the problem, because we're serving the people in the community that have the least resources. And unfortunately, they're going to be prone to be in those places. MATTHIAS STURM: Absolutely. ANTHONY BURIK: Yeah, so all 50 states plus territories have been submitting their state digital equity plans, in anticipation of these federal dollars that are coming in the next few years. So hopefully there was enough commentary about those kinds of situations, Adele. And it's not just in rural communities. We even in urban communities there are gaps, but sometimes they're more pronounced in rural communities. So hopefully enough that commentary has made its way into the state plan, that California will submit to the federal government. And that money will match those needs. And hopefully, within the next year or two, the state will start closing some of those gaps. So we'll keep looking at it from a data point of view. But on the ground, you all know the issues that you're running into in your communities and whether or not they're going to be fixed or not. So yeah. MATTHIAS STURM: Yeah, absolutely. And students are asked, which devices do you or can you use for online learning? Cell phones, less laptops and computers, less tablets. But more and more increasingly, people don't have a device at all, which also would expect the opposite. Hopefully the opposite [MUFFLED VOICE]. What's going on there? AUDIENCE 3: I have a theory on that too. I mean, from my-- ANTHONY BURIK: You want to write the report out that? [LAUGHTER] AUDIENCE 3: Well, the thing is, if you train people that if they say they don't have a device, they're going to get a device, they know they can hock that device. They're likely to say they don't have a device. [INTERPOSING VOICES]. ANTHONY BURIK: [LAUGHTER]. (DESCRIPTION) Slide: Digital Access and Equity (2). Text: Device sharing, data limits, study place, online learning experience. A table with the columns, Sharing with others at home, Data limits that limit learning, Quiet place to study, and Taken online class before. Sharing at home, 2021 to 22, 33.7%, 2022 to 23, 32.8%. Data limits, 2021 to 22, 19.4%, 2022 to 23, 20.2%. Quiet place to study, 2021 to 22, 84.3%, 2022 to 23, 82.9%. Taken online class before, 2021 to 22, 54.8%. 2022 to 23, 46.3%. Text: Figures 12, 8, 13, and 9. Students having to share their device used for online learning, Students' data limits as barriers to online learning, Students' study space for online learning, and Students having taken online classes before. Student Technology Intake Survey Results from 2021 to 2022 and 2022 to 2023 (Source: OTAN 2022 and 2023). Below is another table titled, Items that would help you to study online, with the columns Flexible study times, A device to help me study online, Help with online textbooks and/or classes, Technical help fixing or using online stuff, Help to get on internet, e.g., mobile hotspot. Flexible study times, 2021 to 22, 41.2%, 2022 to 23, 40.7%. Device, 2021 to 22, 27.3%, 2022 to 23, 28.7%. Help with textbooks or classes, 2021 to 22, 24.8%, 2022 to 23, 25.6%, Technical help, 2021 to 22, 15.1%, 2022 to 23, 14.7%. Help to get on internet, 2021 to 22, 14.1%, 2022 to 23, 14.6%. Text: Figure 15. Students' online learning needs. StudentTechnology IntakeSurvey Results from 2021 to 2022 and 2021 to 2023 (Source: OTAN 2022 and 2023). (SPEECH) MATTHIAS STURM: Absolutely, device sharing data limits, quiet place to study. If they've taken an online learning class before, we merged quite a little bit in that table here, in the report, that's for separate tables. But just to show you a little bit of that. So like a third of them are sharing with others at home. I think that's something to keep in mind. Data limits, that limit learning. So if more and more people are using cell phones, that's definitely an issue, if they can't get on Wi-Fi. A quiet place to study, OK, at least. 8 out of 10 to have that. But still, that still leaves 2 out of 10. And students who've taken an online class before, declining Also a little bit surprising. ANTHONY BURIK: Yeah, well, I'm sure we can all get on the soapbox about this one. MATTHIAS STURM: Yeah, absolutely. ANTHONY BURIK: But so much attention during COVID spent on this divide. And if nobody can in-person go anywhere, and the only way to connect with services and people is online, I think I mean, it seems to me that we learned that lesson, and yet the data now is telling us that maybe people are not-- there seems to be some backsliding going on. So anyway, remains to be seen, but yeah. MATTHIAS STURM: Yeah, things that would help you to study online, flexibility is a huge thing in terms of times, other flexibility too obviously. A device, and it was more than a quarter of them help with online textbooks or online classes. Also quarter, technical help is still a big issue. And help to get on the internet for example, mobile hotspot, also quite a bit, but it's holding steady all those, from one program here to the next. (DESCRIPTION) Slide: Professional Development (1). Text: General Technology Use in Education - Proficiency versus Importance. A table with the columns Proficiency Weak, Adequate, and Strong, and Importance Low, Medium, and High. The rows are Acting as a guide for learners when researching online, Integrating technology into daily instruction, Learning how to use new applications (software and programs), Troubleshooting problems that occur when using technology or instruction, Using technology to differentiate instruction, Using technology to manage slash organize my work. The proficiency weak column has percentages from 3 to 6, except for 14 in Learning how to use new applications. The adequate column ranges from 31 to 50. The strong column ranges from 50 to 64, except for learning how to use new applications, at 35. The importance low column ranges from 2 to 7. The medium column ranges from 27 to 33, and the high column ranges from 60 to 69. Text: Figure 16. General Technology Use in Education. C.I.P. Teacher Assessment Survey Results from 2022 to 2023. (Source: OTAN 2023). (SPEECH) And then for example-- and this is from the teacher assessment, we ask or the surveys ask teachers, as you probably know, what they feel they're-- have weak proficiency in or adequate proficiency or strong proficiency. And if that area is of low, medium or high importance to them. I'm sure we could give you just a second to read. And using technology to differentiate instruction. For example, is a Wi-Fi with the highest there, where are we, 69.8, troubleshooting. ANTHONY BURIK: I think one of the things-- in previous reports, we've focused on the difference between that group of folks who feel like they're weak in their proficiency, depending on whatever the technology skills is versus how important they gauge. MATTHIAS STURM: Do you to change the slide? ANTHONY BURIK: Yeah. MATTHIAS STURM: I think that's on the next one. ANTHONY BURIK: Yeah. Well, actually I wanted to just stay on this slide for one second. MATTHIAS STURM: Yeah, sure. ANTHONY BURIK: Just to say that this adequate proficiency, like just the average teacher says, yeah, I can troubleshoot. And so I felt like sometimes when you Ask that question, it's like, never, sometimes, always, that sometimes is quite wide. So is it the low end of sometimes or is it the high end of sometimes? And I kind of feel like in a way, maybe this middle column here on the adequate, is it the low end of adequate or is it the high end of adequate? And so I mean, I think that in this day and age, we would like really the vast majority of people to be strong, to have a strong proficiency with a variety of technology skills. So I know that we've focused on the extremes in previous years. And so maybe we'll take another look at that. But I wonder-- really for me, the divide is right here. It's between either feels weak or just maybe OK. And think about it from the student point of view too, do I want my teacher to be able to solve my problem or not solve my problem? I want my teacher to be able to solve my problem if they can, or the adult educator or the staff member, or whoever it is at the agency. And if it's sort of a 50/50 crapshoot, that might be problematic. We might want to think about building up the educator proficiency across the board in a variety of skills. MATTHIAS STURM: So yeah, absolutely. ANTHONY BURIK: Because everybody has been saying for a number of years now like, yeah, all of these things are really important. And probably more so within the last few years. (DESCRIPTION) Slide: Professional Development (2). Text: General Technology Use in Education - Proficiency versus Importance. A table has columns for weak proficiency and high importance in 2021 and 22, 2022 and 23 each. The rows are: Learning how to use new applications (software and programs), Acting as a guide for learners when researching on the internet, Troubleshooting problems that occur when using technology during and for instruction, Integrating technology into daily instruction, Using technology to differentiate instruction, and Using technology to manage slash organize my work. The proficiencies range from 3.3 to 6.2%, except Troubleshooting problems, which is 13.7 and 14% in the two years. The importance values range from 60.9 to 71.3%. Text: Figure 17. Areas of Weak Proficiency and High Importance to Teaching. General Technology Use in Education. C.I.P. Teacher Assessment Survey Results from 2021 to 2022 and 2022 2023 (Source: OTAN 2022 and 2023). (SPEECH) So yeah. MATTHIAS STURM: Yeah, that's-- ANTHONY BURIK: Yeah, why don't we skip this one? MATTHIAS STURM: Yeah, it's-- ANTHONY BURIK: We kind of covered it. MATTHIAS STURM: Yeah, OK. And we're kind of running out of time, and there's good -- (DESCRIPTION) Slide: What about AI? Text: We are starting to consider using AI in assisting with data analysis, especially with qualitative data, to better capture the voice of the adult education field. What are other ways to use AI for the production of this and other reports at the agency, district, region, consortium level? What are the ethical and other implications to keep in mind when using AI to assist with data analysis and reporting? (SPEECH) Yeah, and then we're starting to think actually today and yesterday -- ANTHONY BURIK: This morning, mere hours ago. MATTHIAS STURM: [LAUGHTER] we've got to say something about AI here. OK, so, yeah, we're starting to consider using AI in assisting with data analysis, especially when we're asking-- it a takes a huge amount of time and the turnover when we get the data. By the time we get the data, and to get the report out to all the open-- and we want to do justice to all the open-ended responses, to turn that over. So there's ways to use AI and not lose-- to better capture the voice of the education field, but not lose that-- of course the human eyes on that data. We don't want to do that. We're wondering what are other ways to use AI for the production of this? And other reports at the agency, districts, region or consortium level. And of course, yeah, we want to try to keep in mind what the ethical and other implications are, when we use AI to assist with data analysis and reporting. ANTHONY BURIK: Yeah, so we don't want-- well, I don't think we want the AI to write the report. MATTHIAS STURM: No. ANTHONY BURIK: We're not saying that. We're thinking about, but how do we start leveraging AI when it comes to all of this data? And not just us, I mean, all of you back at your agencies, like the mountains of data that you're looking at. Are there ways that we can use or start to use AI to get a better handle or provide maybe more objective analysis of the data that we have? So that's what we're starting to think about. But again, the ethical issues that you mentioned. So yeah, I mean, it's-- MATTHIAS STURM: It's a whole new world. ANTHONY BURIK: It is a whole new world, (DESCRIPTION) Slide: Activity - I notice slash I wonder. Text: Select one of the topics for further exploration: 1. Digital Equity and Access, 2. Professional Development, or 3. Distance slash Blended slash HyFlex Learning. Use your topic worksheet to record things that you notice and/or wonder about when you review the Tech and D.L. Report data. Spend some time on your own to review and record, and then we'll get into groups for discussion. Resources: bit dot L.Y. slash T.D.L. 24 folder. There is a QR code on the bottom right. (SPEECH) OK. MATTHIAS STURM: Yes, we still have 20 minutes? ANTHONY BURIK: Yeah, OK. So before we head into the activity, any questions or ideas, comments? And take a look at the online folks as well. I think we're OK. MATTHIAS STURM: So yeah, so we have three topics here that we looked a little bit of data on and stuff like that. And the three worksheets that are in the resource folder. So we wanted to give you a little bit of time maybe. What do you have? We have 20 minutes. Maybe about five minutes just to think about one of the topics, and then we can group together or you can group together with the people who chose the same topic. And then may we just share a little bit what you've been thinking about. (DESCRIPTION) They close the slide presentation and open a Google Drive folder with three Topic worksheets. (SPEECH) ANTHONY BURIK: Yeah so if you're back in the folder, so again, we have the three. So let's just take a look at the topic one worksheet here. So-- MATTHIAS STURM: (DESCRIPTION) They open the Topic 1 document. It has the topic text at the top and a table with columns for I notice and I wonder below. (SPEECH) And of course, yeah, if you can record some of those things, that'll be super for us to know what you noticed and what you're wondering about. ANTHONY BURIK: Yeah, and so the notice really is, when you look at the tables that Mathias has provided, already, Adele and Renee and some of you have already sort of started noticing things about the data. So we would love to get some of those ideas here. And then also, the wonder as well. So when you look at the data, what are the questions that come up for you? What are the things that you think are not addressed or are missing? Adele was talking about her particular issue, about when we administer these surveys, when we get the data back and then how do we look at that data as a whole? Knowing that it's captured at various points in the year, for example, with different audiences. MATTHIAS STURM: And we'll leave the form open for a little while. So if you want to put in notes after the session, that's fantastic. ANTHONY BURIK: Yeah, the wonder too could also be, we're starting to think about AI. You all may be thinking about I bet your agencies, for reporting purposes. So we'd love to hear about what some of your ideas are as well, how we might use AI for some of these things. So we have three different-- good, so someone's already working in the-- MATTHIAS STURM: Yeah. ANTHONY BURIK: Let's come back together as a group. We have about 9, 10 minutes or so left. So before we hear from you, so Mathias and I, we're currently working on-- we're actually working on the report itself right now, the full report. So we hope-- fingers crossed, because again, as Mathias mentioned, some of the data came in late and so we're catching up on all that. So we hope to have the report out-- so this is what? Started March. So we hope to look at a draft version by the end of this month. And then hopefully by April, we'll have the full version. And so as soon as we can get that out to the field, we'll let you know. It will have much fuller sections on the access and equity, the teacher tech skills survey, and then some of the data that has come from CASAS plus the January survey that we administered, to get more information about some of those initial CASAS questions as well. So that's the timeline that we're working on at the moment, but we hope to get that out soon. So we thought we would just want to hear about some of the things that came up in your groups. We can start by looking at the first. Let me just check the chat here. (DESCRIPTION) He opens the Topic 1 document again. The table is now filled in. (SPEECH) OK, let's go to the first topic, topic one, the equity and access. Seems like most people focused on this. Let's just start. Is there anyone online who wanted to share anything that you commented on with the equity and access? And the online folks, you can come on mic, if you would like. Diana, do you want to-- AUDIENCE 5: Sure, I'll go ahead. I was waiting, giving that pause for someone else to have a chance. I put in that I'm just curious about, if the surveys are really distributed to the majority of participants. I'm talking about both staff and students. Or is there a possibility that in some schools, they only give the survey to the advanced level students or the high set, or I should say HSE students. Or is it even across all the students? And same thing with staff, do the tech savvy teachers go in and fill it out? And the ones that shy away from it, postpone it as long as they can or maybe don't even participate. ANTHONY BURIK: OK, good questions, Diana. Thank you. MATTHIAS STURM: Yeah, good question. ANTHONY BURIK: Yeah, so how equitable or maybe equal, but also equitable too, I think the survey question-- AUDIENCE 5: And Anthony, I think you know I've said that for a long time though. Nothing's changed. ANTHONY BURIK: Yeah, and it's good for other people to hear that as well, Diana, so yes, Thank you. MATTHIAS STURM: Thank you. ANTHONY BURIK: OK, what about other folks in the room? Anything that came up with the equity and access issues when you looked at the data? AUDIENCE 8: I just thought that-- well, we were talking about this. So more companies right now are bringing employees back in. I know there were several companies where December was kind of a cut off, and they brought them back in. So I think more and more, there's less people working from home. So just is it no longer worth the cost to pay for Wi-Fi at home, if you're not really working at home anymore. And if you notice, using places in the community has gone up. So if they are working remote or if they're working some sort of hybrid schedule, maybe they're going in the community and not paying for it at home. And that is consistent with some of the other data, like other devices have gone down except for phone, which they probably have Wi-Fi on their phone, but some plans don't include a hotspot. So, yeah. ANTHONY BURIK: So to kind of paraphrase, so things that are happening in the larger world of work may be showing up in the data. Yeah, in that way. That's good. Thank you, Nikki. MATTHIAS STURM: Thank you. ANTHONY BURIK: Yeah Adele. AUDIENCE 2: In the past, and it might have just been in these last two years or one year, I don't recall, but I used to send these surveys out, like I do my principal survey. Because I do a principal survey every month to annoy my students, something like that. But every month, we go by principals letter. Yeah, I only make it four questions though, and I try to use it for program development and stuff, so they see that something happened. But I sent that out after the first time, and I think two things happened. One, they knew what options were available, because they were already students. I wonder, if you're coming in the door the first time, your language is limited. Even though I have someone that probably speaks your primary language, how much does the language barrier impact their understanding of that survey? Number one. And number two, how much does them worrying about, if I say I want in-person or online, is it going to impact me being able to get into something? Because they just don't know enough about my agency, to know that either answer is OK. I think those things, those variables will impact the student's answers. Because I was trying to through that with another agency, and it was weird because we're like what? Seven miles apart maybe. And she had like, only 20% wanted online. And then on my agency, 80% would like to have that option. And I truly think it's a matter of them worrying that if they put one thing, that might limit their access to being able to get into their prospectives. ANTHONY BURIK: OK, thank you. Other equity access issues? Yeah. AUDIENCE 6: So is that pandemic fatigue, where people are just tired of being isolated at home and they want that feeling of just interacting with another human being, might be a reason why they would say no to the online. And they were willing to go into a space, particularly if that space offers other things to support them. So that might be a reason why some of the data also reflects a greater desire to be in face to face, as opposed to online. AUDIENCE 7: Personal connection and good style, like workshop style, orientation style, that way you can just check boxes like you have once you ask the question verbally. And you can collect that by looking at the eye. It's easier. ANTHONY BURIK: OK, yeah. OK, great. Great, thank you. So (DESCRIPTION) He opens the Topic 2 document, which has a few comments filled in the table. (SPEECH) I don't think we got as many comments in the teacher doc. We got a few. Does anybody want to-- just maybe one comment about the teacher skills survey or the teacher PB? Anyone want to share either online or in the room? AUDIENCE 8: Hello, this is Darla. ANTHONY BURIK: Hey, Darla. MATTHIAS STURM: Hi, Darla. AUDIENCE 8: Hi. So I totally agree with your idea that like, we often will choose the middle. Trying to not claim to be strong, if we're feeling pretty strong, but we're not quite there. But I was also thinking of the other end of, many of our agencies have just one person who brings so much to the overall program, but technology is just not theirs in terms of a skill set really. And so I think when we have them take the survey, even though it's fully anonymous, we just always see the results and know that it's not all that anonymous once it comes out. But am from a smaller organization too, so maybe that's part of it. Yeah, which is interesting. ANTHONY BURIK: Yeah, in the smaller agencies, not many places to hide. Right, Darla? AUDIENCE 8: Sure, absolutely [LAUGHTER] We tend to know each other better, yes. ANTHONY BURIK: Got it, OK. Thank you. (DESCRIPTION) He opens the Topic 3 document, which has several comments in the table. (SPEECH) All right, and then how about again in the blended learning distance? Maybe just a comment, if someone wanted to share something they talked about in their group or online or in person. It would be great, somebody's still writing. AUDIENCE 9: And one little thing. We both have high school diploma kids. ANTHONY BURIK: Is it the same? AUDIENCE 9: I mean, we are trying to give them their diploma. They're self paced at their homes, if they are. And that even-- like she said, they have the access, they're 24/7 on the phone. But that darn Google form is not answered ever. Like that is the new generation, also like ignoring the stuff. Not even like third world country coming here to US, this is a new thing for them. No, they were born here. They know the system. They know Google form is the way. We know it, everybody. Just 10 little questions. Can you please say, yes, no, yes, no? No, they are resisting. I don't know why. How can we reach them? Those people are just kind of-- choose to be hidden like that. That's it. They know the questions are about their high school diploma goal. When are you graduating? And then how much are you willing to-- give us program, like in two hours a week or something. No, they're just ignoring it. No, that's like teenager style. OK, you just yell at them. Dinner is ready, nobody is coming. That's it. That is another piece that we're dealing with in person. ANTHONY BURIK: Sure, yeah. It's interesting. I mean, using the technology to really make things very open and accessible, and yet you as you say, you're facing this kind of resistance. Right? Or maybe, I'm just going to do the bare minimum. I'm not really going to make full use of it. But you want to have potentially a very robust, blended program or a HyFlex program or hybrid program. And yet, where is the buy in, right? Yeah, OK. So I know that we're at time. So thank you for these comments. Mathias and I will take a look at these. This is really great feedback just to hear. Also, we're getting the report together, and its final phases here. So again, thank you for that. So I think that we'll basically go ahead and fill out the evaluation. But like I said, yeah, Mathias and I are working on the report right now. (DESCRIPTION) Slide: Evaluations. Text: Please Submit Your Evaluations. On the right are two QR codes. The top is, All Presentations. https:// T.D.L.S. dot O.T.A.N. dot U.S. slash Evaluation slash. The bottom is Symposium. https:// T.D.L.S. dot O.T.A.N. dot U.S. slash Evaluation slash Conference slash. (SPEECH) Like I said, we hope to get it out soon, within the next month or so. And again, we appreciate all of you taking the time to read it, take a look at it. If you have questions or other ideas-- and again, to Adele's point earlier, hopefully this report and other reports like it are really helpful for you at your agencies. That's one of the goals as well, is that you can actually make use of it for your planning purposes, your set goals, whatever other goals that you're working on at the agency. So I think Mathias, anything to close with? MATTHIAS STURM: No, thank you again. Thank you for coming. ANTHONY BURIK: Yeah, thanks a lot. (DESCRIPTION) Slide: OTAN Online. Text: Technology and Distance Learning Symposium. California Adult Ed Technology and Distance Learning -- A Look at the Data. Presenters: Matthias Sturm, Adult Education Researcher. Anthony Burik, OTAN. March 2, 2024. Slide: OTAN, Outreach and Technical Assistance Network. Text: Follow Us. Twitter, slash O.T.A.N. LinkedIn, slash company slash O.T.A.N. underscore 2. Like Us, Facebook, slash O.T.A.N. Serves Adult Education. Subscribe, YouTube, slash O.T.A.N. Serves Adult Education. Text, Professional Development, News, Teach with Technology, Videos, TDLS, Online Resources. Website, O.T.A.N. dot U.S. -- Phone, 916-228-2580.