(SPEECH)
[AUDIO LOGO]

(DESCRIPTION)
Logo: A dark blue circle contains the white silhouette of a person in its middle.

(SPEECH)
OTAN Outreach and Technical Assistance Network.

(DESCRIPTION)
Text: Leading adult education through support for and the effective application of technology. In the middle of the screen is the OTAN, symbol surrounded by a multicolored circular border that contains illustrated faces of different people, each in their own colored segment of the border. All around this circle is text that reads: Professional Development. Teaching with Technology. OTAN, Digests and Newsletters. News and Social Media. Annual Technology and Distance Learning Symposium. Online Resources and Video Presentations. Website: O.T.A.N. dot US. Phone number: 916-228-2580. Text on next screen: OTAN, Online. Canvas - Assessing Outcomes and Standards. Presenter: Cory Chitwood, Principle Learning Consultant, Instructure. October 10, 2024.

(SPEECH)
CORY CHITWOOD: My name is Corey. Some of you-- I've been working with OTAN for several years now, so I might have had the pleasure of working with some of you already before.

(DESCRIPTION)
Slide: Cory Chitwood. A photo depicts Cory from the waist up. He wears glasses and has a full beard. Text: Years at Instructure: 6. Years in education: 23. Superman. Real housewife. Player. Broadway baby. Music educator. Travel enthusiast. Utah resident. Kentucky native. Loves peanut butter, nachos, and being a husband. Bachelor of Music Education and Master of Music from Eastern Kentucky University. Wow. Cory was a breath of fresh air. He is very excited about Canvas; it shows in his face, his speech, his comments and his actions. I learned a great deal about Canvas from him. He has a way of making you want to take the deeper dive into Canvas - he makes us feel confident and safe, He really is a Canvas Superman!

(SPEECH)
I was a middle school choir teacher for 21 years who used Canvas in my classroom. I was also a Canvas administrator for my school district. And I am a principal learning consultant now for Instructure. So if you like Clarissa, I'm sure you'll like me too, because I'm the one who trained her. So there it is.

(DESCRIPTION)
Slide: Accessing Beta Account. A table chart with columns labeled Task; Production; and Beta, and rows labeled New Features; Refresh Cycle; Deploys (Bug Fixes); Email Notifications; and Testing LTIs is depicted. In the address bar above the chart is the website yourschool.beta.instructure.com.

(SPEECH)
So before we begin, there's a few things I would like to get your-- I need your input. So before we even get into what's on the screen right now, last week with Clarissa, the agenda was to learn how to create outcomes, organize outcomes into outcome groups, uploading outcomes with CSV files.

The question that I have is, did you at all talk about rubrics? That's one of our bigger goals for today. So can someone just put a yes or no in the chat with me that was here last week. Yes meaning we did talk about rubrics or no meaning I don't think we did talk about rubrics.

I just want to make sure that I'm not doubling up. Good. Perfect. Thank you very much. I just want to make sure I wasn't going over things that Clarissa might have found time to go over herself. So now that we've established that we're on the same page, now, you might already know about your beta account if you want to practice. I'm sure you've been told this every single week.

So if you feel-- if you're not already logged in, if you want to log in to your beta with outcomes. I really don't feel like it's all that essential to log in to your beta account. No one's really going to see them or use them or find them until you even tell them they're there. So you can always delete what you're doing.

(DESCRIPTION)
Slide: Session Objectives. A bullet-point list of this session's end goals.

(SPEECH)
So today, you've already done bullet point one, which is create and upload outcomes and manage them in their accounts. So today, our focus is now that you have outcomes in your account, how will teachers assess those outcomes?

Well, there are two different options. Number one is adding outcomes to your rubrics and attaching those rubrics to different types of assessments. Another option is aligning outcomes to our quiz questions.

(DESCRIPTION)
Slide: Session Expectations. On the top right corner is clip art of a computer. Text: I will show you something. I will demonstrate how to do it.

(SPEECH)
So we'll take a look at all of that today. I'm just like, what I'm sure is that been the case for the past four sessions. I'll give you time to try everything out today. Please don't feel pressured to flip between screens and do everything as I'm doing it. I will give you plenty of time to try out everything that I'm showing you today.

But please don't hesitate to ask any questions you have in the chat, please make sure that your chat is set so that everyone can see your questions. And if you ever need more time on anything, please let me know that as well.

(DESCRIPTION)
Slide: Account and Course Rubrics.

(SPEECH)
So as I was saying, now that you have outcomes in your account, the way that those outcomes are assessed are through rubrics and quizzes. Teachers can create their own rubrics and use those account level outcomes in their rubrics.

Or if-- and I don't really know that this will apply to anyone here, but if there are any common assessments in multiple courses that are going to use the same rubric, you can create those rubrics at the account level and then teachers can just when they build their assignments, find that rubric and use it.

So if there are outcomes in that account level rubric and then teachers find that rubric and use it, then you will have outcomes reporting for every single course that use that rubric and assess those outcomes.

So again, really the only benefit of that is if you have any common assessments that is going to be delivered across multiple courses. Otherwise, again, teachers will create their own rubrics, but then add those account level outcomes to their rubrics. And you still get reporting for everywhere those outcomes were used. You're just not doing the work in creating that consistency for teachers.

I'm going to skip through all of these so that we can just talk about it. So as I was saying, we have two options. Here, I am at the account level.

(DESCRIPTION)
On the screen he navigates to a browser tab labeled rubrics. On the left-hand side of the page is a blue side navigation bar with the admin icon highlighted, and the admin menu visible. To the right, account rubrics are listed.

(SPEECH)
I am-- I've logged into my admin menu and I've gone to rubrics in my admin menu. Over here, I'm in a course.

(DESCRIPTION)
He clicks on another browser tab labeled Cory's Metropolis U Sandbox. On this page, the courses icon on the blue navigation bar is highlighted and its menu is visible.

(SPEECH)
And when I go to rubrics in my course, it looks exactly the same.

So again, either I can create rubrics at the account level that any teacher at my site can use or teachers can create their own rubric, but still use those account level outcomes. Let's talk about both.

And then you can practice wherever you want. If you practice at the account level or practice inside of a course, it's the exact same process. Everything looks exactly the same. The only difference is if you create the rubric in a course and you want to add outcomes to that rubric, you do have to import your outcomes into that course first.

(DESCRIPTION)
He hovers his cursor over the outcomes option in the Courses menu.

(SPEECH)
So again, I don't know exactly what it looks like at your site and I don't know exactly if there would be a need for account level rubrics due to, again, multiple courses, multiple teachers, giving the same assessment and using the same rubric.

So I'm going to actually work here at the course level. So here I'm in a course, I'm in my Sandbox. But it could be one of my real courses. And I'm going to first import the outcomes that I am going to assess.

So I'm actually going to go to outcomes in this course, first.

(DESCRIPTION)
He clicks outcomes in the courses' menu and opens the outcomes page. A menu labeled Outcome Groups appears onscreen. In this menu he clicks the option Demo Outcomes, and a list of all demo outcomes appears to the right of the menu.

(SPEECH)
So this is how teachers are going to assess those outcomes that you created last week with Clarissa.

(DESCRIPTION)
He selects some groups of outcomes from the drop down menu under Outcome Groups. He navigates to the three dots above each group that opens in the upper right corner and selects remove from the pop-up menu and then clicks the red remove group button in the confirmation dialogue. He navigates to Cory's Metropolis U Sandbox Outcomes via the Outcome Groups menu.

(SPEECH)
Just trying to clean up a little bit. Sorry about that.

So if I'm a teacher in a course and I went to assess some of those outcomes that you all created last week or have created since, I'm going to come up here to find.

(DESCRIPTION)
He clicks the gray find button in the top right corner. This opens a page titled, Add Outcomes to Course. To the left is a sidebar menu labeled Outcome Groups. Under this menu is account standards. He clicks this and a dropdown menu appears.

(SPEECH)
And then I see the account standards. Metropolis University, that is the name of my institution. So that's where I would go. Of course, you would probably see either OTAN or the name of your site.

(DESCRIPTION)
He clicks Metropolis University and another dropdown menu appears.

(SPEECH)
And then here, I have the California College and career readiness standards.

(DESCRIPTION)
He clicks this and to the right one item is listed under the California College and Career Readiness Outcomes.

(SPEECH)
Looks like I only have one. Anyhow.

So when teachers go in to import the outcomes or standards that they want to assess, it's important that they only import the outcomes that they're actually going to assess. You don't want a whole library of standards or outcomes, whichever verbiage you like to use, that you're going to have to constantly search through and sift through to find the outcomes that you're actually assessing.

So over here, I have the South Carolina State standards.

(DESCRIPTION)
He clicks this option from the Outcome Groups sidebar menu then clicks Academic Standards in the dropdown menu that appears. To the right a long list of Academic Standards Outcomes appears.

(SPEECH)
I wouldn't want to pull at a too high of a level. So that again, I'm not assessing hundreds and hundreds or I don't have hundreds and hundreds of outcomes.

And if you do try to import a large amount of outcomes, it will give you a warning, saying, hey, you're importing a lot of outcomes. Are you sure you want to do that? Only import the ones that you're actually going to assess. So I'll use this California, even though it didn't, I only seem to get one. I want to add that outcome.

(DESCRIPTION)
He navigates back to the single California and Career Readiness Outcome and clicks the gray add button in the top right corner.

(SPEECH)
Now that outcome now exists in my course lives in my course and I can add that outcome to any rubric that I want or align that outcome to any quiz question and new quizzes. And sometimes you might depending on how all of your outcomes are grouped and organized, you might have to come to one group and add the outcomes that you're assessing.

Then go to another group and add the outcomes that you're assessing. Then go to another group and add the outcomes you're assessing. It really just that's something that you really want to be thoughtful of as you're building and organizing your outcomes.

(DESCRIPTION)
Cory clicks a yellow button at the bottom right corner of the page and navigates back to the previous Outcomes page where Cory's Metropolis U Sandbox Outcomes are listed. On the Outcomes Groups menu on the left hand side of the list, he selects Leadership and Digital Learning from the drop down menu.

(SPEECH)
So in there, I have that one outcome from leadership and digital learning that I imported from my California career readiness. This is not a real California career and college readiness outcome. It's something that I just made up last time I was training.

So that's step one. First, teachers are going to come to outcomes and find the account level outcomes that they're going to be assessing in their course. And then the next step is to create the rubric that I'm going to put the outcomes in.

So there are two ways that you can create rubrics. You can go to rubrics in your course menu

(DESCRIPTION)
He hovers the cursor over the rubrics option in the courses menu that is opened from the blue side navigation bar.

(SPEECH)
or you can create an assignment and then create the rubric there with the assignment.

To be honest with you, my personal preference is to create the rubric with along with an assignment. But because normally I don't think that far ahead. I don't think oh, I have an assignment come up. I haven't even created the assignment yet. But let me think about the rubric. That's just not the way that I do things.

But, again, if you with your, especially, with your California career and college readiness standards, you might have some rubrics that you reuse on a pretty regular basis. And so creating that rubric ahead of time might make perfect sense. But I'm actually going to do it the way that I like it. And that is by creating a rubric attached to an assignment.

So I'm going to go to Assignments.

(DESCRIPTION)
In the courses menu, Cory clicks assignments, and to the right, a list labeled, Daily Assignments, populates in the middle of the screen.

(SPEECH)
And I'm going to create a new assignment, I believe, I don't think I have a practice one here that I want to use. So I'm going to create a new assignment

(DESCRIPTION)
He clicks a yellow plus assignment button at the top right of the screen. A form with tabs labeled Details and Mastery Paths appears in the center of the screen. Under the details tab the form has several fields some of which include assignment name, points, and a drop down menu for submission type. At the bottom right of the page are three buttons labeled cancel, save and publish, and a yellow save button.

(SPEECH)
and I'm just going to call this ESOL oral proficiency assignment with rubric. Just so that I can identify it.

And then I'm going to fill out my assignment like I normally would. Choose what assignment it's going to be. And then save it or save and publish it. If I'm not ready for students to see it yet, then I'm just going to save it. If I am ready for students to see it, I'm going to click Save and Publish.

And then when you save your assignment, the plus rubric button is in the bottom left corner.

(DESCRIPTION)
Once he saves the assignment, the page is titled E.S.O.L. Oral Proficiency Assignment (with Rubric) and a description box appears below the title. On the bottom left of this form is a grey button labeled Rubric.

(SPEECH)
So I'm going to pause for just a moment to let everyone get caught up. I'm going to ask you in a course, whether it's your Sandbox course or whether it is one of your real courses, I want you to first go to outcomes and find those account level outcomes that maybe-- this is just for practice.

So if you made some practice outcomes last week with Clarissa, then you can import those.

(DESCRIPTION)
He clicks outcomes in the courses menu, then clicks the find button in the top right corner.

(SPEECH)
If you were not here last week and you have no outcomes in your account, then today just create a rubric without outcomes for practice and then you can go back and check out last week's video and create those outcomes and try it, again.

(DESCRIPTION)
He clicks assignments in the courses menu, then scrolls through the list of assignments that opens up in the center of the page and hovers over the E.S.O.L. assignment he just created.

(SPEECH)
So step one, again, come to outcomes. Find those outcomes that you created last week. Then go to assignments and create a practice assignment that you want to practice with the rubric and save it. Now, I don't actually have that text on my screen. Let me see if I can unhide some slides I probably do up here.

(DESCRIPTION)
He navigates to the browser tab labeled Admins Outcomes and Competencies which brings up his slideshow. On the left sidebar he scrolls through his slides and chooses a slide titled Create or Find Outcomes with a timer below the title and instructions on the right side of the slide. He edits the instructions in real time.

(SPEECH)
So this is not identical but it'll work.

So again, I'm going to-- regardless of the text on my screen, I'm going to ask you to import your outcomes or find your outcomes from last week into a course. Of course, you don't have to name it and edit and all that stuff. Then you are going to create a fake assignment to add a rubric, too. So find an outcome, create a practice assignment maybe online file upload type of assignment, and then save it.

Again, if you ever feel like you need more time on any of this, please just let me know either in the chat publicly or privately, whatever, so that we can make sure that you feel good about this process.

(DESCRIPTION)
He navigates back to the browser tab labeled, E.S.O.L. Oral Proficiency Assignment which brings up that assignment. After he clicks the plus rubric button, a form appears under the assignment with a field for title; boxes for criteria, ratings, points; and checkboxes next to several other settings. ****

(SPEECH)
After we have created this assignment, we're going to click this Plus Rubric down here in the bottom left. Now if I created the rubric in my rubrics menu first or if I am looking for an account level rubric that we've discussed for the past 20 minutes, then I'm going to click Find a Rubric.

(DESCRIPTION)
He points his cursor to the hyperlink text, find a rubric, in the upper right corner of the form located under the assignment.

(SPEECH)
When you click-- so whenever you create a rubric as a teacher, that rubric is associated with you the user not associated necessarily with the course. So any rubric that you create in any course is always available to you by clicking on Find a rubric.

(DESCRIPTION)
After he clicks on find a rubric, a pop-up box labeled, Find Existing Rubric, appears in the middle of the screen. On the left side of the box is a list of courses and the number of rubrics each course has.

(SPEECH)
Here I see all of the courses where that I've created rubrics at-- 65 rubrics Oh my goodness that's a lot. It shows me how many rubrics are in each course. As you saw a moment ago, my institution is called Metropolis University. So there are my root account rubrics that, again, maybe the Canvas administrators made for me to use for those common assessments.

So once you create a rubric, you never have to create it again. And whether I created it in my rubrics menu or if I created it right here along with this assignment or if it is a count level rubric, no matter what, I'm always going to be able to click Find a rubric.

So this is our ESOL oral proficiency rubric.

(DESCRIPTION)
He closes the Find Existing Rubric pop-up box, and types into the title field of the form located under the E.S.O.L. assignment.

(SPEECH)
And then when you create a rubric, you have two options of criteria. You can create your own criteria. That's different than the outcomes that you're assessing.

For example, let's say that I had my students create a slide presentation and I am assessing-- some of our outcomes or standards are being assessed with the information and skills that they're learning by creating this slide presentation.

But I also want to score them on the details of the slide presentation. Maybe the rules were you have to have a minimum of five slides. Maybe the rules were you have to include these bullet points of information in your slide presentation. Maybe the rules were you have to present it to the class and it has to last at least seven minutes long.

Those are specific criteria for this assignment that has nothing to do with our outcomes or standards. Again, the outcomes or standards are what did they learn or what did they show proof of learning or what skills did they get better at? Whereas, again, showing that they used at least five slides and presented for seven minutes is really just following the rules.

(DESCRIPTION)
In the criteria box of the form, he clicks the pencil edit icon in the top right corner, and a pop-up box labeled, Edit Criterion, appears in the middle of the screen. It has fields for description and long description. He fills in the fields, then clicks the yellow, Update Criterion button in the bottom right corner and the box closes.

(SPEECH)
So if I do want to give them a specific grade for their presentation for the class and then I will say the presentation followed all of the required criteria, which are five slides or less or more, seven minute presentation, And required criteria listed in the assignment instructions.

So maybe this is worth 50 points.

(DESCRIPTION)
He types in 50 points in the points box of the form, and the number of points in the full marks section of the ratings box updates to match it.

(SPEECH)
And that's going to update my top rating of 50 points. But then to edit this and to explain to my learners what they did to earn all 50 points, I'm going to click on that Edit Pencil.

(DESCRIPTION)
After he clicks the edit pencil located at the top right corner of the Full Marks section of the ratings box., a pop-up box labeled, Presentation for the Class, appears in the middle of the screen. It has fields for rating score points, rating title, and rating description.

(SPEECH)
I can change the title from full marks to something else if I don't like full marks. And then I will say these student included all of the REQUIRED Information with no errors. That's pretty generic, but no errors.

And then I'm going to use this plus sign to create maybe a 40 point rating.

(DESCRIPTION)
He clicks the yellow, Update Rating button on the bottom right of the pop-up box and the box closes. He clicks the blue plus sign located between the Full Marks and No Marks sections of the ratings box. A pop-up box labeled Presentation for the Class appears in the middle of the screen with fields for rating score points, rating title, and rating description. He fills in these fields, then clicks the yellow, Update Rating button in the bottom right. The box closes and now this section has been added to the ratings box of the form.

(SPEECH)
And I'll call this meets expectations. And I will say the presentation included all of the required criteria, except for the length of the class presentation. Maybe if it was five minutes, I'm not going to be too mad about it.

(DESCRIPTION)
He clicks the blue plus sign located between the Meets Expectations and No Marks section of the ratings box of the form and proceeds to add another section in the same manner as before.

(SPEECH)
And then I'll add a 30 pointer here. I'll call this below expectations. And then I'm going to explain that the presentation was missing at least two important elements. And then I will do a 20 pointer. We'll call this remediation required. And then I will say the presentation was missing at least three required elements.

And then I'll do a 10 pointer and I will maybe we'll call this reassigned and we will say the student must do the presentation over, again. Again, I'd probably want to come up with better verbiage than that. But nonetheless. And if I want to talk about what 0 means, I can do that too.

Then I want to talk about-- I want to also assess those skills that they showed that they've gotten better at or that they've learned from my outcomes. So then I click plus outcome.

(DESCRIPTION)
To the right under the criteria, ratings and points boxes is a magnifying glass icon labeled Find Outcome. Cory clicks this and a pop-up box labeled Find Outcomes appears in the middle of the screen. On the left-hand side of the box, he clicks a folder labeled Leadership in Digital Learning. He then clicks a link labeled CCR - Anchor 1 and a points scale appears to the right.

(SPEECH)
And then as I mentioned, I'm only going to see the outcomes that exist in my course. That's why I had to import them first. So here is that leadership and digital learning outcome that I had earlier.

Now outcomes cannot be edited. So however-- on whatever point value and whatever scale you created that outcome at the account level, that's exactly how it's going to look at the course level. And it's not editable.

That way we get consistent data. If multiple teachers use the same outcome and everyone rates it on a different point value and a different scale, then we're not getting very good data. We're getting very scrambled data. So the reason why outcomes are not editable is so that we're getting consistent reporting, no matter who is assessing this outcome.

By default, this check mark is checked.

(DESCRIPTION)
Under the points scale is a checkbox next to text that reads: Use this criterion for scoring.

(SPEECH)
And what that means is that when I assess this outcome in the rubric, in addition to feeding their learning mastery score, they're also going to earn up to 5 points on this assignment for achieving different levels of mastery. So they can earn up to 5 points, depending on how well they've shown mastery of this outcome.

(DESCRIPTION)
With his cursor, he points to the far left box of the points scale labeled Exceeds Mastery, 5 points.

(SPEECH)
So again, if I uncheck it it doesn't give them 5 points on their assignment. But I'm still tracking learning mastery of these outcomes and I can still see the reporting in my Learning Mastery Gradebook, which we'll talk about and in your account level reports, which we will talk about.

But again, with it being checked, it means not only am I checking, am I giving them a mastery score, I'm also giving them up to 5 points on this assignment.

(DESCRIPTION)
He clicks the yellow import button at the bottom right and the pop-up box closes.

(SPEECH)
And again, notice that outcomes can-- they don't have the edit pencils, they can't be edited at all.

(DESCRIPTION)
The CCR - Anchor 1 outcome with the points scale has been added to the E.S.O.L. form under the criteria, ratings, and points boxes.

(SPEECH)
And you can add as many outcomes as you want. And you could add as many criteria as you would like.

(DESCRIPTION)
The cursor hovers over the plus criterion hyperlink next to the find outcome magnifying glass icon.

(SPEECH)
Now here is why I prefer to create my rubrics with assignments. When you create a rubric with an assignment, you see these assignment settings.

(DESCRIPTION)
He points with the cursor to the settings with checkboxes at the bottom of the form.

(SPEECH)
When you create your rubric in your rubrics menu, you won't see these settings until you attach it to an assignment and go into Edit mode of that rubric.

A lot of extra steps. So I think it's just easier if I haven't already created the rubric to create the rubric with the assignment, so that I automatically have these rubric settings. Now these rubric settings, the reason why we don't see them over here in the rubrics menu is because these rubric settings mean, how do you want to use this rubric with this specific assignment?

Because sometimes we want to use rubrics in different ways. Sometimes we want to give our learners a grade with our rubric. Sometimes we want to just give feedback with our rubric. Sometimes we want to track learning mastery of our outcomes with our rubric. Sometimes we want to do any combination of those three.

So these settings do not transfer when you use this reuse this rubric. Because the next time I reuse this rubric, I might only want to use it to leave feedback. And then the time after that when I reuse this rubric, I might want to use the leave feedback and give them a grade. And then the time after that I might want to leave-- use it to leave feedback, give them a grade and track learning mastery of some outcomes. So that's why.

So here are our options. First option is I'll write free form comments when assessing students. And so that removes all of the criterion ratings. And instead you would type in your feedback and manually type in points.

(DESCRIPTION)
Cory clicks the first checkbox and all previously entered points information is removed from the ratings box, and the points scale is removed from the CCR - Anchor 1 outcome row. He then unchecks the setting and the information reappears.

(SPEECH)
If you want to use this rubric just to leave feedback and not to award any type of grade, you can remove the points from the rubric.

(DESCRIPTION)
He clicks the second checkbox, and all number points are removed from the ratings box and points scale, but the performance descriptions remain.

(SPEECH)
If you have outcomes or standards in the rubric, but you don't want this particular assignment to count towards their learning mastery score, then you can choose, don't post the results to the Learning Mastery Gradebook.

If-- this is very important. If you are multitasking right now, please stop what you're doing so you don't miss this. Very important. Alarm, alarm, alarm. If you want to use this rubric in SpeedGrader, which all teachers should, if you want to use this rubric and SpeedGrader and award a grade in SpeedGrader, check this box.

(DESCRIPTION)
He checks the setting: Use this rubric for assignment grading. When he checks this box, the setting under it that reads, Hide score total for assessment results, disappears. When he unchecks the box, the setting below it reappears.

(SPEECH)
If you don't check this box, you can still use the rubric in SpeedGrader, but you have to take the extra step of typing in their grade. But if you check the box, all you do is click in the rubric criterion ratings and then it will automatically give them a grade as well. So don't forget that people tend to forget that. And then don't understand why they have to still type in a grade and SpeedGrader.

So I'm going to select Create rubric.

(DESCRIPTION)
At the bottom left of the form is a yellow create rubric button. When he clicks this, a pop-up box labeled Change assignment points to match rubric? appears in the middle of the screen. At the bottom right of the box are two buttons labeled Change and Leave Difference.

(SPEECH)
It's going to tell me that my assignment is worth 100 points. My rubric is only worth 55 points. What do I want to do about it? So I'm going to change my assignment point value to match my rubric point value.

(DESCRIPTION)
He clicks the change button and the pop-up box closes.

(SPEECH)
So in the future, if you-- if I'm going to reuse this rubric on another assignment.

(DESCRIPTION)
In the courses menu on the left-hand side of the page, he right-clicks assignments and selects open in new tab. A list of assignments appears in the new tab.

(SPEECH)
So let me create a second assignment really quickly. I'll do it real fast.

(DESCRIPTION)
He clicks the plus assignment button in the top right of the page. The assignment form opens up and he names it 2nd assignment. He then clicks the yellow save button at the bottom right.

(SPEECH)
So when I went to reuse that rubric on another assignment, I'll click on Find a rubric.

(DESCRIPTION)
With the page now open to the 2nd assignment, he clicks the gray plus rubric button towards the bottom left and the same form demonstrated earlier with a field for title; boxes for criteria, ratings, points; and checkboxes appears. He clicks on Find a Rubric towards the top right of the form and the pop-up box labeled Find Existing Rubric appears in the middle of the screen. He scrolls through a list of courses on the left-hand side of the box.

(SPEECH)
I'll look in this course I'm in my Sandbox right now. Where is my sandbox? It's probably-- this one right here, quarries.

Yes. There's my ESOL oral proficiency rubric. I'll just select Use this rubric.

(DESCRIPTION)
He clicks the gray, Use This Rubric button towards the bottom middle of the box and the box closes.

(SPEECH)
But I'm going to have to go back in, click the Edit pencil, and notice, again, it did not keep my settings.

(DESCRIPTION)
He clicks the edit pencil located at the top right of the form, and a confirmation dialogue appears at the top of the screen. He clicks ok. All of his settings are unchecked.

(SPEECH)
So with this second assignment, I still want to, again, click on the Edit pencil and then choose use this rubric for assignment grading.

(DESCRIPTION)
He moves up to the top right of the page and clicks the edit pencil to edit the entire 2nd assignment. It takes him back to the form page where he names the assignment. He clicks the three dots at the top right of this page, clicks delete, and clicks ok in the confirmation dialogue box that appears at the top of the page.

(SPEECH)
So now, I'm going to give you a chance to get caught up.

(DESCRIPTION)
He switches to the browser tab labeled Admin Outcomes and Competencies and, after navigating through a few slides, stops at one titled Create a Rubric with a timer below the title and instructions that read: 1. Create a Rubric and add at least two criteria with descriptions. 2. Adjust the ratings columns. 3. Find and align an Outcome to the Rubric. Decide on some Rubric settings and select "Update Rubric."

(SPEECH)
So again, if you want to work at the account level, if you want to work in the rubrics menu of your course level, or if you want to create a rubric right there with an assignment, those are your three options right now. You can do all three-- any of those three.

And then, we will take a look at adding rubrics to quizzes. And then we'll finish up with learning mastery data with the Learning Mastery Gradebook and with your account level reports. Again, if you would like me to show you something, again, or if you have any questions, please let me know.

(DESCRIPTION)
The timer counts down from 5 minutes.

(SPEECH)
The second way that we assess learning mastery of our outcomes is through quiz questions,

(DESCRIPTION)
He navigates to the browser tab labeled, E.S.O.L. Oral Proficiency Assignment and on the courses menu clicks quizzes. To the right, a list of assignment quizzes appears.

(SPEECH)
and that is through new quizzes. So when you go to a course and go to quizzes-- and you can also-- I mean, let me rewind a little bit actually.

So again, there is the option. If you don't want teachers, again, if you're going to have common assessments where multiple courses with different teachers are going to be offering the same assessment, you can create a count level item banks. So here I am at the admin level.

(DESCRIPTION)
He navigates to the browser tab labeled Rubrics. On this page, the Admin icon is highlighted in the blue side navigation bar. He hovers the cursor over Item Banks in the admin menu.

(SPEECH)
You can create a account level item banks. And then as you are creating those questions, you can align those questions to your standards and you can share those item banks with all of your teachers. And those that are assessing those questions can import them into their quizzes. And the admin has done all of the work for them.

(DESCRIPTION)
Now he navigates back to the browser tab that contains the Assignment Quizzes

(SPEECH)
But, again, if you don't have any common assessments or if you're just going to let teachers deal with create their own, when you create a quiz, you do-- you can only align quiz questions to New Quizzes. So if you're given the option of New Quizzes versus Classic Quizzes, it does have to be New Quizzes.

(DESCRIPTION)
He clicks the yellow, plus quiz button at the top right of the page. A pop-up box labeled Choose a Quiz Engine appears in the middle of the screen. He clicks the yellow submit button at the bottom right of the box. A form for the new quiz appears.

(SPEECH)
And then you're going to give it a name.

(DESCRIPTION)
In the assignment name field, he types in Practice New Quiz.

(SPEECH)
And then at the very bottom-- I should see a plus rubric button down here. I don't know why I don't. Maybe they moved it. So when you create a quiz and go into the build menu--

(DESCRIPTION)
He clicks the yellow build button at the bottom right of the page. It opens a page titled, Build, and a box that is titled, Practice New Quiz for OTAN, appears towards the top of the page. There is a blue plus sign at the bottom middle of the box.

(SPEECH)
When you create new questions here with this plus sign, no matter what question type you choose, you can align outcomes that are already in-- and, again, they have to already be inputted in your course. But all question types have a section that says align to outcomes.

(DESCRIPTION)
When he clicks the plus sign, a box opens labeled Insert Content. It has several options such as multiple choice and formula. When he clicks an option, a form with multiple sections opens up.

(SPEECH)
This is multiple choice. This is formula. All question types have a section that says align to outcomes, and it really is just as easy as clicking in this section.

(DESCRIPTION)
When he clicks on align outcomes it takes him to a page titled Attached Outcomes. In the middle of the page is a bullseye with an arrow pointing to it that has Align Outcomes written under it. Cory clicks on a folder to the left that is labeled Leadership in Digital Learning. To the right a hyperlink to the CCR - Anchor 1 outcome appears. He clicks the checkbox next to it then appears to click a button in the bottom right corner of the page.

(SPEECH)
There I want to find that outcome, again. Just check it and select Confirm alignments. You can align multiple outcomes. Here I have many of them. I could align multiple outcomes to the same question as well.

(DESCRIPTION)
He clicks on a folder to the left labeled Demo Outcomes and a list of demo outcomes appears in the middle of the screen. He checks several of the boxes next to the listed outcomes.

(SPEECH)
I don't have an activity about that planned, but if you have any questions or want to show me that, again, or want me to show you something, again, please let me know and I'll be glad to do that.

(DESCRIPTION)
He navigates back to the page with the assignment quizzes list.

(SPEECH)
Let me delete that one I just made.

(DESCRIPTION)
He finds the practice quiz he made in the list, clicks the three dots to the right of the entry and clicks delete. He confirms the deletion in the dialogue box.

(SPEECH)
Now, where do we see all of the information? So at the course level, teachers will go to their Gradebook-- actually, let me pull up a slide. I don't know if I have one in here.

(DESCRIPTION)
He navigates to the browser tab where his slide presentation is located and navigates through his slides. He stops briefly on a slide titled Instructor Learning Mastery Gradebook before returning to the browser tab with the assignment quizzes list.

(SPEECH)
So teachers will go to their Gradebook-- let me go to a different course. There we go.

(DESCRIPTION)
On the courses menu he clicks grades.

(SPEECH)
And they will turn on-- oh I don't have the Learning Mastery Gradebook. Hold on just a minute. Wrong course. Sorry about that.

(DESCRIPTION)
He clicks the courses icon on the blue side navigation bar and a list of published courses appears on the right side of the menu. He selects concert choir from the courses. The published courses list disappears and the original courses menu reappears. On this menu he clicks grades.

(SPEECH)
There we go. To grades.

(DESCRIPTION)
A table with students' names and categories they are graded in appears. To the top right of the page is a hyperlink that says Gradebook. He clicks on this.

(SPEECH)
And then to the Learning Mastery Gradebook and the dropdown menu.

(DESCRIPTION)
The link takes us to a table labeled with students' names and the amount of points they have in five different categories.

(SPEECH)
Now, all of my students right now are overachievers. They're all exceeding mastery. So here is what it would look like more normally.

(DESCRIPTION)
Cory navigates to the browser tab that contains his slide presentation. He presents a slide titled Instructor Learning Mastery Gradebook. Text: The Learning Mastery Gradebook helps instructors and admins assess the outcome standards being used in Canvas courses. This gradebook helps institutions measure student learning for accreditation and better assess the needs of their students. To the right is a table similar to the previous one, but with the amount of points each student has more varied than in the last example.

(SPEECH)
As you hover over each column, it'll show you a pie chart of how well students have achieved mastery of those of the different outcomes that have been assessed in this course.

(DESCRIPTION)
He navigates back to the browser tab labeled Learning Mastery containing the first table of students and their points earned.

(SPEECH)
I love the Learning Mastery Gradebook because it allows you to toggle on and off different mastery levels. So that you can focus on students who are at different levels of mastery.

(DESCRIPTION)
To the right of the table is a panel with four different levels of mastery: Exceeds Mastery, Mastery, Near Mastery, and Below Mastery. Each level is color-coded. There is a checkbox that allows you to hide outcomes, and an export report option.

(SPEECH)
You can also hide the outcomes or standards that you've imported into your course, but you haven't assessed yet, and you can export your Learning Mastery Gradebook into a spreadsheet.

So again, you just go to your traditional Gradebook and in the dropdown menu you should see Learning Mastery Gradebook. Also with quizzes, I forgot to mention with quizzes, if you want to see how well students achieve mastery of the outcomes that were assessed just in that quiz--

(DESCRIPTION)
He opens an new tab and navigates to the Canvas login page. He logs in and his account opens up on his dashboard page.

(SPEECH)
I'm going to open up a quiz for you right now. So I can show you.

(DESCRIPTION)
To the left of the page he clicks on courses on the blue side navigation bar. The heading at the top of the page reads Canvas Trainer Course and several icons appear in the middle of the screen and he clicks on the New Quizzes icon. A list of new quizzes appears and he clicks on one titled Thursday Afternoon Science Quiz. The form for the quiz opens and he clicks the blue build button at the bottom right of the page. A page opens up and at the top of the page is a horizontal pane with several tabs.

(SPEECH)
So in quizzes, as I was saying in the Reports tab, there is an outcomes analysis report that shows learning mastery of just the outcomes that were assessed in that quiz. Let me see if I have a screenshot of that. If not, I'll go back and I can show you one. I'm going to pull this screenshot up for you real quick. There it is.

(DESCRIPTION)
Screenshot of a table with students' names in the far left column and a row of outcomes at the top.

(SPEECH)
So there's the outcomes report and New Quizzes. So real quick and easy to see. Britain is doing very well. This outcome itself is doing pretty well. This one down here is doing pretty well. Got a lot of Black in here that we need to maybe do some remediation.

So that is so far we've got learning mastery scores for the quiz, we've got learning mastery scores for the course, and the Learning Mastery Gradebook. And then finally, you have learning mastery-- well actually not finally, learning mastery scores for individual students.

(DESCRIPTION)
He navigates to the browser tab labeled Gradebook Concert Choir and clicks the gradebook hyperlink at the top left of the page and selects Learning Mastery Gradebook. This takes us back to the first table showing students names and their points earned in five different categories.

(SPEECH)
If you go to people in a course or if you go to your admin menu and go to people in the Admin menu, either way, if I'm in my admin menu, I can find that student.

(DESCRIPTION)
At the center top of the page is a search box. He types in the name Grant and that student's basic information appears on the page.

(SPEECH)
Click on their name and view their outcome results right here.

(DESCRIPTION)
After he clicks on the name more information appears such as login information and courses enrolled in. At the top right of this page is a hyperlink that reads: See Outcome Results for Christian Grant.

(SPEECH)
If I'm a teacher of a course, I can go to people in my course. And then down here at the three dots go to User details.

(DESCRIPTION)
He navigates to the browser tab that is open to the people page from the courses menu. He finds Christian Grant's entry. After clicking user details, a page with Christian's information opens, and to the right is a menu with the option Outcomes Report.

(SPEECH)
And I have their outcomes report there.

So let me repeat that. If I'm an admin, I can go to people in my admin menu, search for anyone with the student role, and see their outcome results here.

(DESCRIPTION)
With the cursor he points to the hyperlink at the top right of the page that reads: See Outcome Results for Christian Grant, and a list of outcomes appears on the page.

(SPEECH)
For all of the courses, every outcome they've ever been assessed on.

If I'm in a course, I can go to people, go to user details, and then view the outcome results report for just this course.

(DESCRIPTION)
The report has columns for outcome, attempts, latest score, and average percent.

(SPEECH)
So I can see how many times they were assessed on a different outcome, I can see what their most recent mastery score was, and I can see what their overall score is.

If I want, I can hide the outcomes that have not yet been assessed.

(DESCRIPTION)
To the right of the page is a menu with the option to hide unassessed outcomes

(SPEECH)
And if I want to see where they were assessed, I can click on Show All artifacts.

(DESCRIPTION)
Show all artifacts is located above the hide unassessed outcomes option. When clicked, it opens a list under the outcome report.

(SPEECH)
And that will show me-- it shows me the assignment and what their score was.

So again, at the course level I'm only going to see outcomes that were assessed in this course. If the account level I want to see all of the outcomes for that student for all of their courses, whether they've been assessed or not. If the outcome exists in that course, it'll look like this.

(DESCRIPTION)
He points with the cursor to the outcome report on the page.

(SPEECH)
And then as an admin, the last thing that you have. And this really is the last thing. I think, let me double check. Let me make sure I didn't leave anything out. We got the Learning Mastery Gradebook. Yes. We've got the outcomes report and New Quizzes report. So yes, this is the last thing.

(DESCRIPTION)
He navigates to a browser tab where the admin menu is open. He navigates to the account settings page and a horizontal pane at the top of the page has several tabs, including a report tab.

(SPEECH)
So in your admin menu, if you go to Settings, which is at the bottom of your admin menu, and go to Reports in your Admin Settings, there are three reports related to outcomes. Outcome export, outcome results, and student competency report.

If you click on any of these, it'll give you a description of what that report is and an example of how that report could look.

(DESCRIPTION)
A pop-up box labeled Outcome Export appears with a table with several columns of information.

(SPEECH)
But the outcome export literally just gives you a spreadsheet of whatever outcomes exist in your account.

It doesn't give you any learning mastery scores or anything like that. It's just a spreadsheet version of the outcomes that exist in your account. And last week with Clarissa, if you were there, you learned how to upload a spreadsheet of outcomes. So this is how to export a spreadsheet of outcomes.

The outcome results report is really the report that you're going to use. It's going to show you all of the outcomes in your account and how well learners are achieving mastery of that outcome.

Per student, you'll have a column, per student to show them-- show you their scores for how well they're achieving mastery on all of the outcomes, where the assignment was, all of that great stuff. Of course, it's just going to give you a spreadsheet that you can filter out and manipulate however you want.

The student competency report. To be honest with you, I have been working with Canvas since 2018. I've been using Canvas since about 2016 and I have never understood the value of the student competency report, and no one has ever been able to give me an explanation that makes sense to me.

The reason why is because the student competency report does not include data from learning mastery data from quizzes. It only includes learning mastery data from rubrics. So if rubrics are the only way that you are assessing learning mastery, if you're not using quizzes at all, then this is a great report to you. But again, if outcomes are being assessed in quizzes, the student competency report will not include that.

So a real quick review of what we did today and then we're going to head out. We started off-- you built your outcomes last week. This week, I showed you how to import outcomes into a course.

This week we talked about you can create account level rubrics that if teachers are giving common assessments, you can create them at the account level. If not, then teachers are going to first import their outcomes and then create their rubrics and their course.

(DESCRIPTION)
He switches to the tab where the courses menu is open and points with the cursor to the quizzes and then the rubrics options in the menu.

(SPEECH)
You can create rubrics in your rubrics menu or you can create a rubric directly attached to an assignment. If you create-- either way, you want to make sure to always check your rubric settings and to make sure you use this rubric for assignment grading as checked.

We looked at our learning mastery data in the Learning Mastery Gradebook and the quiz outcomes report and the student outcomes report and then these three account level reports.

(DESCRIPTION)
He navigates to the browser tab where his presentation is located and presents a slide titled Outcome Results and Competency Reports in an Account. A bullet point list reads: Outcome Export, Outcome Results, and Student Competency Report.

(SPEECH)
So that's our review of what we've accomplished and a little bit over an hour. Before we head out, are there any questions or was there anything that you would like for me to show you again?

(DESCRIPTION)
A slide titled Wrap up is presented onscreen. Two QR codes are in the middle of the screen. The code on the left has text that reads: Receive your Badge! If this is your first webinar, and email will prompt you to create a Free Canvas Badges Account. The code on the right has text that reads: Take our Survey! We would love to hear from you! Website: https://bit dot L.Y. slash P.L.S. survey link.

(SPEECH)
If not, if you want to earn your badge for attending today's training, you can use the QR code on the left. If you feel like leaving a survey about your experience through all of these cohorts, don't forget to give Clarissa a shout out. You can use the QR code on the right or use the link that short bitly link on the right.

Thank you, both of you, for hanging out with me the whole time. I am-- if you by chance come back for cohort two, it'll be the exact situation, but it'll be me instead of Clarissa. So if you feel like you need to just go over some of the information, again, I'm pretty sure it's OK for you to come back for any of the second cohort sessions. Thank you both so much. I hope you have a great day and hopefully I'll see you, again.

(DESCRIPTION)
Text: Follow us: X slash OTAN. .. LinkedIn slash company slash OTAN underscore 2... Like us: Facebook slash OTAN Serves Adult Education... Subscribe: YouTube slash OTAN Serves Adult Education.