[music playing]

Speaker: OTAN, Outreach and Technical Assistance Network.

Matthew Rhoads: All right. Welcome, everyone. Thank you for joining me for this presentation on using coaching cycles to create data visualizations for decision-making. So really, what we're going to be talking about today is how you can take interaction from teachers that you're working with in terms of observations, coaching, et cetera, professional learning. And how can we take that data and put it into really nice visuals so that you and stakeholders can make decisions for the betterment of your teachers, students, et cetera. So that's the goal of today's presentation.

A little bit about me. I work for Education to Career Network of North San Diego County. I do all the tech integrations. So I work as a coach, do all the professional learning, work with about 100, probably 100, 120 teachers and at the consortium level, along with a number of our other shared positions. So it's fun working in a lot of different schools. All the ESL teams, CPE, ASE, ABE, all the programs, I work with.

And so I'm also a professor at San Diego State University in a dual language, English language learner, program, as well as at Macquarie University Irvine, where I work with doctoral students. And so really, what I'm going to be talking a lot about today is just stuff that we do every single day in our consortium to all of our schools that are partners.

And I gave you a sheet. There's information there where you can take notes, as well as a QR code that can take you to my flow page that has everything about me. And then for those, actually, that are watching this, I'm going to give them the link to this presentation real quick and putting it in the chat. I did this earlier. [ INAUDIBLE ] There we go. Perfect. Slideshow is now in the chat.

Audience: Thank you.

Matthew Rhoads: All right. So why instructional coaching? So it's funny, in California, in K-12, it's been utilized. But in a lot of other parts of the United States, instructional coaching, you use quite often. In California, and especially Adult Ed, I think it's truly in its infancy. It's different in terms of a lot of the instructional leadership. It's based on the value of [ INAUDIBLE ] versus just building the capacity of our teachers. And investing in instructional coaching, there's a ton of research behind it and how there's a large effect size to impact student learning and outcomes, as well as teachers' overall efficacy in their practice.

So I want to outline a number of studies real quickly just describing the effect of instructional coaching. That's not evaluative. It is for really impacting student outcomes and teachers overall [ INAUDIBLE ] asks you to teach better. And we notice in this study right here by Kraft and Blazar, where we look at the effect of coaching.

It's a 0.49 effect size, which is actually quite large when we look at-- if you know John Hattie's work and his effect sizes, instructional coaching with this effect size is actually quite high. And we can really bridge a gap between novice and veteran teachers by that instructional coaching and really see over time here with the achievement that student coaching programs have produced larger improvements in teacher practice and show larger effects on student achievement.

And this is done in the K-12 environment. But the results are being really picked up by a lot of schools throughout the country, especially in the Eastern coast and Midwest, where they-- for example, a school site may have two or three instructional coaches along with their staff. And districts may have about 10 to 15, that they're working with teachers constantly.

And what it says over here on the far right, larger programs are less effective. So for example, if you have a coach covering more than 100 teachers, a single coach, then you're not going to have that much effectiveness. You're want to have a coach cover a smaller number of teachers because that's where they can make that impact. And you'll see why in a minute when we talk about coaching cycles and what goes into that.

So just wanted to share this study here, you can check it out if you want, as well as three other major studies regarding instructional coaching and its impact on teaching Ed learning. It's truly, I think, really emerging as a really powerful way to build capacity, but also, like I said, build teacher effects advocacy, as well as ensure that they don't burn out.

So what we did in our consortium is we started with our goals in mind. So in our three-year plan, we discussed the need for professional development. And really, we get into the nuts and bolts of it right here, staff participation, and training, and coaching related to diversity, equity, and inclusion. Integration of technology, and classroom, and structural strategies are promoted. So that's where it comes from.

And also, regular use and access professional development website, serve as a central location for opportunities for professional learning. And we really wanted to focus really specifically on students who engage in active learning within our class offerings. So we wrote this within our three-year plan. And that's the basis for what we're doing.

And we found a measure for wanting to measure technology integration as well as active learning within our classrooms. So we ultimately decided to use a technology integration matrix, also known as the TIM. And that's through the University of Florida, South Florida, I believe, or Central Florida. And it's a really cool matrix, where you can see-- let me just pull this up here. Let me see where I can pull up really-- I want to see-- show you guys a big image of it. There it is.

So what this does is it measures technology integration through four-- five different steps from entry level all the way up to transformational. And we start out with active, collaborative, constructive, authentic, and goal-directed. And what we do here is we're measuring, what exactly are we seeing in classrooms based on this? We go in the classrooms, and observe, and provide feedback, as well as say like, hey, we think that this teacher is adopting Google Slides at a conventional level that's procedural that's used in their classroom.

And what's cool is that we can sit down with the teacher or can sit down with some leadership team and talk about, what does this all mean? And I can show examples of what this looks like in classroom environments. So we're utilizing a rubric that is something that we can measure what we're seeing with, but also show examples of what that is. And we can provide what that coaching looks like. So that's just a little bit of our rubric. So I have it all here.

So when we-- when I came into this division two years ago really was to build relationships with teachers because, a lot of teachers, when they don't know what instructional coaching is embedded within their job, they think it's like, someone's coming in to evaluate me. It's like, hey, he's coming in to really judge what I'm doing versus, hey, this is my partner in here.

I'm going to help them work on something that they may want to learn as well as what we want them to learn. So building relationships with them is key in developing a culture where this is the norm within our consortium. So that took me about six to nine months to do when I was setting this all up.

So really, this is how we created this culture of instructional coaching is that explain the positives, not evaluative. It's collaborative. We're learning together. The walking coach, so I go to all of our classes. Throughout the week, I go to various school sites. I try and drop in just as many classes as possible. Five to 10 minute drop by's provide a little bit of feedback that is an email sent to them, as well as I'm tagging various things of what they're doing. And they can see that in the email that I send them. And then, that goes to our database.

And then, what that does is ultimately leads to coaching sessions because I get to see them more and they know what I'm doing. And we talked a little bit about what they're doing in the classroom. Sometimes, we have follow-ups after I come in. Or sometimes, they want to schedule a meeting with me, do a one-on-one session when I come into their classroom or a Zoom session.

So these are some things that help build it. And then, consistent communication. We have a newsletter that goes out for all structural practices and tech integration. We have an open line of communication through just texting me and email. We have asynchronous professional learning on our ETCN EdTech website, where we have video, slides, thousands of templates that's all there in one location so that they can access everything that we're working on, as well as any training we've had.

So then, we also offer the synchronous professional learning, where we have series. For example, right now, we're working on the 2023 spring active learning UDL and differentiation series, where it's a four-part series, where we provide a synchronous consortium-wide offering that is recorded.

And then, the teachers get paid to come. And then, they get paid to do the tasks in the class. So for example, they need to build it out. And they need to have students interact with it. And then, I check to make sure that they're doing it. And then, they get those additional hours. So the payment piece for the synchronous learning has been really effective.

And then, we also had to discuss like, what does this all mean for us as a consortium? So I had to really talk to stakeholders about why we're doing this, starting with the research and then collecting the data to drive exactly what we're doing in these trainings based on what I'm seeing, and then aligning it with goals for our coaching and professional learning. And those were the goals that we've come up with. So it's developing that ongoing culture where this exists.

And then, talking about the implementation of coaching cycle. So coaching cycle is essentially, you're working with a teacher throughout the school year or a designated period of time. So you first identify what they're doing. And then, you and the teacher work together so that they can learn and move the needle in that area.

What we're doing is evaluating them in the coaching piece with the TIM, the matrix. And then, we're looking at examples of them improving. And then, we look back and identify it. So you'll see what that looks like in a minute. And by Jim Knight. And this is-- I've utilized Jim Knight's framework for this. So this is just what I've just mentioned. So now, this is the collecting the data piece and showing what this looks like. And then, the visualization so then we can make decisions based on what we see.

So for example, I can type in all my different types of observation types, which lead to, what am I working on with the teacher? Is it just a strictly coaching session? Is it just normal tech support? Are we working on our Canvas course? Does it lead into a professional development session? Co-teaching and planning, so that leads into opportunities where I go and teach them something together in a class with them.

And then, I do tags. So what do I see that's happening in that class when I come in? And I tag it here. And we have tags. And it can be even more sophisticated. I've changed up the sophistication of the tags based on each strategy and tool. So I can see how many times this is happening and what we're working on, new trends. And I can see how much time I'm engaging with educators. And I can break that down per hour. And I can show you what that looks like in a second.

Audience: Sorry, when you say tagging?

Matthew Rhoads: Yeah

[interposing voices]

Matthew Rhoads: So it's just like creating a tag. So for example, I'll liken it to, say you're writing an email to someone. And you can create a-- a tag is essentially just what that conversation could be about. So in the system that we utilize is that we're tagging what these things are about. Or what are we observing? What are we coaching about? So that we can keep track of those data points. That's what tagging is. That's what it is. And I'll show you--

Audience: Do the trending tabs sit within the interaction types?

Matthew Rhoads: Yes.

Audience: So are they a subset of the interaction type?

Matthew Rhoads: Yeah.

Audience: And then, the interaction types are within that TIM?

Matthew Rhoads: Yes. Yeah. Yep. And then, we can look at how we're supporting various buildings based on the school. When are we doing it? And how much time are we investing into them? So we can get-- making sure that, based on their teacher population, that we're trying to distribute my time as much as possible. And then, who am I doing it the most with?

And across here, this is mostly just-- some of these are designated CTEs. So generally, for us, I'd say about 60/30, 60% ESL, 30% CTE, and then 10% everything else. And then, we can see throughout the year when the most interactions are occurring throughout the week based on category.

And this is what the coaching cycles look like. So say we want the teacher to work on active learning. And it's based on the TIM matrix. We could have had a five. And this is, for example, in the beginning rating. Then it's after a six to eight week process, oops, that they have, that they've improved based on observation, feedback, number of times that we've put in our coaching cycle, that they moved the needle up. Same thing if a teacher needs to work on assessment or if they need to work on Canvas using Canvasing tool.

And then, we can see across the board, I've done about 50 cycles. And then, I can determine which educators are making growth, which are making considerable growth. So my goal is each year to focus on, give or take, about 50 teachers. I can't invest so much time into everyone.

But if I can focus on 50 that I'm doing significant amount of work with, then I know that we're really going to move the needle with those people, and especially the ones that are more enthusiastic, than maybe the bottom 10, 20% [ INAUDIBLE ] would be the laggards. But the goal is just that you can make the considerable growth with the adopters, hopefully, that more trickles down to the laggards.

Audience: How would you define considerable growth?

Matthew Rhoads: So considerable growth is, for example, if they move up one. So if they move up one whole scale piece of the rubric. So if they started at one and they moved to a two, that would be considered considerable growth. 0.5 would be just normal growth [ INAUDIBLE ] And then, we can see across the board here-- I took out names for this.

But say this is the coach's name. You can see-- say this person's focus is on Canvas. We can see their initial rating was one. Now they're over a two. Their growth is normal. This person was one. They were now a three. They moved up two. And we can see this within the database. And we're using it based on the rubric here, the five-point rubric.

And this is just my philosophy is that the more informal feedback that you're giving, the more that leads into coaching sessions, the more that leads into course design, the more that leads into more professional learning. So the more. And there's actually a psychological-- a lot of research relating to this. The more constant access or more people see you, the more likely they're wanting to do something with you. So visibility. I think it's called-- it's a visibility factor. So ultimately, that's my philosophy with the features that we have in our consortium.

And this is where I do the data collection piece. So within connecthub.io-- or you can do this on a Google form. And this is the person's name. At the very top, I type in their name. And it could have it already in our database. And then I have, what's my interaction type? What's going on? What's the narrative piece? Then I start adding tags based on what we're doing or what it is. And then when I'm done, if it's an observation and formal feedback, I send them an email. And it has exactly whatever [ INAUDIBLE ] And it goes right to their email. And then, that goes into our database.

Audience: [ INAUDIBLE ]

Matthew Rhoads: Yep. And what we've done recently with one school in our consortia, they wanted to do more full on learning walks. So they've adopted this. The administrators have adopted the tool and what I'm doing. So they're doing constant learning walks with this and collecting the data for their site level. And then, they're using that to monitor their progress and make those decisions based on where their teachers are at instructionally. So this is how we collect data.

And then, the power of the visualizations is like, hey, we know what we're doing. We know we're tagging. We know what the trends are. We know where these teachers are at based on where they're at on the rating scale and where we need to grow. And then, we can also look at other data. So for example, like cost of testing or high school testing, can we maybe see if there's a correlation between the work that we're doing and those outcomes? So that's why this is really important because we're collecting at this other data set that we can use, then compare to see if it's moving the needle with their students.

Audience: [ INAUDIBLE ] have you done that yet? Are you doing that?

Matthew Rhoads: We are doing that. Yeah.

Audience: I would suspect that would be interesting to see.

Matthew Rhoads: We need it more longitudinally. So I need to see more over the next couple of years.

[interposing voices]

Matthew Rhoads: What can I see after like three or four years of this? And then see over the course of a couple of years, is there that huge measurable growth? But the studies say in K-12 that there is an opportunity for considerable growth.

Audience: Yeah, that's-- I think we've all got that in our hand. And training, we get that. Do you get to go to your district-wide train-- administrator trainings that have nothing to do with adult Ed? Well, no. We've never had to go. And so we get-- so a benefit, we get to hear some of this stuff. And we do the learning walks. But this makes it help for the teachers. Or at least, it makes it easier to understand and know. So that tagging is when you're actually doing observation, writing everything. And while you're writing, you tag this [ INAUDIBLE ]

Matthew Rhoads: You're tagging. Yeah, you're tagging everything. And you can determine, so say you have a rubric that you want to use. Maybe it's not the TIM. It can be [ INAUDIBLE ] It can be [ INAUDIBLE ] It can be really whatever. You can put those into the system and then use those tags based on what your goals are.

Audience: Gotcha.

Matthew Rhoads: Yeah.

Audience: And you guys still do training, those that are doing the walk-- well, you're doing-- you're the expert. You're doing the walkthroughs. Do you-- do other schools use other staff, like someone that's trusted, they know? Obviously, not the administrator.

Matthew Rhoads: So only one school currently is doing the full on learning walks. I'm doing it for the consortia. And I don't share, for example, personal names. I just share what the performance looks like--

Audience: Gotcha.

Matthew Rhoads: --across that school. And if they want, I can share it with them.

Audience: Gotcha.

Matthew Rhoads: But I try to keep it-- there's a fine line you don't want to cross at times. So you have to navigate that in the position that I'm in at least. So this is year one. This was year one. I wasn't doing coaching cycle. But then, I moved to year two with coaching cycle. So now, we can see year-to-year progress. So that's really a cool piece of this. And these are some of the tools that you can use. I use Connecthub. I know you can use it on Google Forms and Sheets. Connecthub is only $10 a month. And so we've invested in them. But you can set up the same thing with Google Forms.

Audience: Can I ask, why Connecthub?

Matthew Rhoads: Connecthub is just-- I think if we're looking at coaching platforms, it creates all this data visualization without you having to do it on-- for example, if I fill it out in Google Form, then I can go to Sheets. And then, I would have to create the visualizations on Sheets.

Audience: Right.

Matthew Rhoads: This creates it for you.

Audience: Right.

Matthew Rhoads: So that's--

Audience: And it's $10 a month for how many--

Matthew Rhoads: So for example, it's $10 per user.

Audience: Oh, OK. So if there was 50 teachers, [ INAUDIBLE ]

Matthew Rhoads: No, no, no.

Audience: Oh.

Matthew Rhoads: So if it's just me, it's $120. And it's just based on the coaches, the number of coaches. Yeah, so if it's an administrator, that's $120. There's another coach, it's $120. Or if it's another admin, it's $120. But what you can do is you can-- everyone can be on one account together. And you can then split your work collaboratively and add more data to the system.

Audience: And so your admins, or admin coordinators, or whoever is evaluating teachers are also using [ INAUDIBLE ]

Matthew Rhoads: So one school is doing it for more of the evaluation piece. The other schools have been given the option. So they're considering whether to do that. But we're doing this, though, for building capacity for the consortium.

Audience: Have you looked at Power BI?

Matthew Rhoads: PRBI?

Audience: Power. It's a Microsoft product that-- so I think it's the same idea. Visualization tool. We pull data from wherever.

Matthew Rhoads: Yeah, is it like Tableau?

Audience: I don't know. I don't know. I haven't used Tableau. And I don't really use [ INAUDIBLE ] we have somebody that does it for us. But he pulls all our data together and then presents it however--

Matthew Rhoads: Yeah, you could do-- very much, you could have it on-- all this on a spreadsheet and have them do it that way as well.

Audience: Yeah.

Matthew Rhoads: It just really depends. And then what we do, we can easily have-- I send quarterlies, semesterly, and yearly data reports to all of our administrators based on where we're at. So I send them an email every quarter. And they can review it. And we want to focus on the growth of teachers in relation to a rubric, because that's associated with our goals, and then align the data with the yearly professional development plan, so our three-year plan, or the schools under lots of accreditation, or that kind of thing.

Audience: And which consortium do you run again?

Matthew Rhoads: Education to Career Network of North San Diego County.

Audience: [ INAUDIBLE ]

[interposing voices]

Matthew Rhoads: Yeah, and--

Audience: So you're working with both college and K-12 adults schools.

Matthew Rhoads: But we're not really focused on much of [ INAUDIBLE ] college. We haven't gotten too much onto their campus.

Audience: Yeah, I guess

[interposing voices]

Matthew Rhoads: Yeah, not-- well, because of [ INAUDIBLE ] I won't say it on a recording. But you know why.

Audience: Yeah, gotcha.

Matthew Rhoads: So before we go, I want you to jot down on the piece of paper here, keep this in mind, I used to think versus now I think. So write down some key themes and ideas. And do you have any question-- write down any questions that you have. And then, what's your next to-do list? Are you going to look up some more information about this? Do you want to look at some research? Do you want to talk more about it? So go ahead and jot that down right now. And for that.

And then for, I think, whoever is online right now, you can jot that down in the chat if you want. Or you just can jot it down on a piece of paper. So just three key themes and ideas, two questions or wonderings that you have, and then one extension with this thought that, what did you used to think about this? And then, now what do you think about this? So go ahead and spend some time. All right. Does anyone have any thoughts? Or what were some of your main takeaways from this? And do you have any questions?

Audience: Yeah, I have a question.

Matthew Rhoads: Yes.

Audience: Can you give me a specific example of an area that you identified in a teacher's-- when you observed in a classroom and you said, OK, this is something that could be improved. And then, what was the improvement you suggested? And then, how was-- how did that impact student outcomes, student experience?

Yeah, so for example, this happens all the time. So say there's a teacher that's providing someone a direct instruction, where they're talking to somewhere from the class and [ INAUDIBLE ] They're just passively listening. So I've gone in. And I've observed and looked at their lecture, looked at the slides, et cetera. And I talked a little bit about, did you-- I ask certain questions. Do you consider-- when your students are listening, are they not doing anything on their notes? If they're doing things interactive. I use that type of language. I say, are you considering this? I go like a question.

And then in my observation, here's some ideas to start considering because then you may have students be a little bit more engaged, more have [ INAUDIBLE ] more discussion. I can provide a short little strategy like, maybe I would like you to consider using more pair share. After you're provided a little bit of direct instruction, have your students do a short little pair share and bring it back to the class.

Something simple, that can change-- then if we're thinking about tech, maybe you want them to have a little bit of an interactive Google slideshow, where the students are actually writing down their notes on a Google slide or describing their thinking. And then, that can lead to more pair shares and conversation. And then, you as a teacher can see what your students are thinking in real-time. And then, you can monitor and adjust what you're trying to do. So those are just some basic things that we talk about.

Or it can be even simple, something like Canvas. Does your-- is your Canvas friendly to-- does it provide video to help upload content? Does it have a slideshow? Are you collect-- are you conducting any assessment on your Canvas course? So those are just some of the questions that I sometimes start off with. [ INAUDIBLE ] is usually just one.

But then, I provide a lot of positive reinforcement. I do four or five positive reinforcements followed by a couple of questions. And then, that leads to the conversation. And then, it's just a matter of showing up all the time, and taking interest, and then building that relationship. And then, that ultimately will have them be more motivated to want to engage with you, and have a conversation, and then work in-- have coaching [ INAUDIBLE ] or go to professional development, or both.

And then, you start [ INAUDIBLE ] over time, you start seeing those outcomes. So you know in terms of what their class looks like. If you go enough, you can see over time, those different things that they're trying, maybe they have more confidence. Or maybe that they want to try some new stuff. And then you cater to, how are they best going to learn? What does this person's schedule look like? I can give them a bunch of templates. Or I can give them, watch this video online or come to this professional development.

So you try and meet them where they're at and give them options and choice, as well as you're supporting them in what you feel like that they may want to improve in. And there may be a lot of conversations like, oh, yeah, maybe I do talk too much. Or maybe I need to make this more interactive. Or maybe I need to try and learn how to use those tools and strategy. A lot of times, after a couple of conversations, they're like, oh, yeah, I think I agree with you.

And sometimes, you're going to run into conversations with people where they're not going to want to change. But then, you maybe try and focus on things on the fringe. So that's ultimately my philosophy and how I've done it over the past couple of years. And I think if you can get-- hit about 80% of the teachers that are wanting to be coached, then you're going to have that organizational change versus-- you can't hit everyone. It's going to be impossible.

But hopefully, like I mentioned earlier, the trickle down effect. If you get to adopters, you can get people that are excited about it, that it can change to departmental change. So we just, for example, at one school said that we're focusing on UDL. And we wanted to incorporate on Canvas a page for-- digital literacy page, where they would first watch a number of videos related to skills, like Google Docs, slideshows, et cetera.

And then after they've reviewed the digital content, then they would be asked to complete a number of tasks. And the students were given a choice on how they wanted to demonstrate their learning. So they could write. They could write the steps of how to use this tool. They could record themselves talking about the steps using this tool. Or they could do something [ INAUDIBLE ] They could write a number of steps. But they could also talk about how they're doing it. How are they using that particular tool to help them with their digital literacy?

So that's just one example. And the team was very enthusiastic over time. And it took a year to get them really enthusiastic. But now they're like, oh, I can see how this is really benefiting students. This is benefiting us. And also too, on campus, you can collaborate. So they're building out pages. And they're dividing the labor. So they don't have to do as much.

So I think when you're focusing on those pieces, it really does help. It's not going to be perfect. But I think persistence and consistency will-- is the key to this. So if you have any questions, please feel free to reach out. Hope to connect on social media. You can check out my blog. I do have a podcast, so if you want to learn more.