IMPACT Live!

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?

Thanks for sharing, Luke! I've been on a journey over the past year to improve assessment in my courses too. Your interest in making it a better experience, perhaps even a learning experience, is really incredible. I'm curious about the logistics of your tests: how did you manage meeting with all of your students and how many standards or topics were you able to assess at a time? Also, could you share how you communicated progress to students? I'd love to know what your feedback looked like and whether or not students received that feedback in real time. 

Thinking more broadly, I'm curious if, and in what ways, other instructors use reassessments in their courses. Luke wouldn't need to invent a new system! At a later time, students could test again with a new problem to demonstrate learning. I'm curious if more instructors are using grading as a way to increase learning in this way so that feedback loops become a positive experience for students. 

------------------------------
Drew Boso
Math Instructor
Northwood Technical College
Ashland WI

Original Message:
Sent: 10-06-2023 15:03:06
From: Luke Walsh
Subject: Curiosity in Assessment

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?

------------------------------
Luke Walsh
Instructor
Catawba Valley CC
Hickory NC
------------------------------

Greetings:

I've been using oral assessments for a couple of years.  However, I prefer to call them "interviews" to avoid the connotations of "tests".

The workload is OK for me, since I teach year-round with a reduced load.  It's important to note that interviews may not be suitable for every class (see below).  I rely on interviews for my math for liberal arts class, and I use them as part of the assessment of my statistics courses.

Here are some of my observations:

The interview environment may not be well-suited for every population of students.  Students on the lower levels often have more anxiety about interactions with the instructor, which can affect their performance for an interview.

At the same time, interviews can be a useful option for those students who work better in speaking compared to writing.

I've found that it is vital to address the emotional well-being of the student, and to guide them into a feeling of safety, and collaboration.  Part of that process is focusing on how students learn, e.g. having them bring their notebook of class notes and homework to the interview.  I emphasize students' ability to explain how to do a problem, so it's important to have those resources there.  This practice also allows me to provide guidance about better ways to take notes.

I also provide a general set of questions before the interview, to help students identify what subjects they need to cover.  Typically, I cover five questions during a 30 minute session, and this is repeated four times per term.

During the interview, if a student has not prepared at all, or hasn't brought their notebook, then we will set up another time -- at that point, they have a zero. If they are completely lost on a question, then they can choose to take a zero on that question, or we can work through it together.  If we run short of time, then we schedule another interview time. 

I have perhaps a unique perspective on the assessment -- if I can help clear up a student's misunderstanding during the interview, then I will give them full credit for that question.  I rely on other types of assessment (online work, take-home tests) to provide a well-rounded skill assessment.  Since I offer opportunities to redo work on an exam, it makes sense that I can address concerns during the interview time.  

It also takes some work for me to develop skill in interviewing.  Interviews can be especially challenging in an online class, doing video conferences, when I may not have established much of a relationship with the student.  One time, I made the mistake of insinuating that a student was cheating, since they were having trouble answering questions.  Not a good idea!  There are many reasons for a student having difficulty with responding, so it takes patience and skill to navigate that.

In my opinion, the engagement and depth of understanding achieved through interviews is much richer than the standard testing environment. While it involves some risks, I believe the benefits outweigh the difficulties in most situations.

------------------------------
Rick Powers
Instructor
Western Technical College
La Crosse WI
powersr@westerntc.edu

Original Message:
Sent: 10-07-2023 01:22:44
From: Drew Boso
Subject: Curiosity in Assessment

Thanks for sharing, Luke! I've been on a journey over the past year to improve assessment in my courses too. Your interest in making it a better experience, perhaps even a learning experience, is really incredible. I'm curious about the logistics of your tests: how did you manage meeting with all of your students and how many standards or topics were you able to assess at a time? Also, could you share how you communicated progress to students? I'd love to know what your feedback looked like and whether or not students received that feedback in real time. 

Thinking more broadly, I'm curious if, and in what ways, other instructors use reassessments in their courses. Luke wouldn't need to invent a new system! At a later time, students could test again with a new problem to demonstrate learning. I'm curious if more instructors are using grading as a way to increase learning in this way so that feedback loops become a positive experience for students. 

------------------------------
Drew Boso
Math Instructor
Northwood Technical College
Ashland WI

Original Message:
Sent: 10-06-2023 15:03:06
From: Luke Walsh
Subject: Curiosity in Assessment

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?

------------------------------
Luke Walsh
Instructor
Catawba Valley CC
Hickory NC
------------------------------

Thanks for sharing, Luke! I've been on a journey over the past year to improve assessment in my courses too. Your interest in making it a better experience, perhaps even a learning experience, is really incredible. I'm curious about the logistics of your tests: how did you manage meeting with all of your students and how many standards or topics were you able to assess at a time? Also, could you share how you communicated progress to students? I'd love to know what your feedback looked like and whether or not students received that feedback in real time. 

Thinking more broadly, I'm curious if, and in what ways, other instructors use reassessments in their courses. Luke wouldn't need to invent a new system! At a later time, students could test again with a new problem to demonstrate learning. I'm curious if more instructors are using grading as a way to increase learning in this way so that feedback loops become a positive experience for students. 

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?

Yes thank you for sharing, Luke! 

Before answering your question, I have two questions of my own! :)

My Questions - Were students given the opportunity to choose between an oral exam and a written exam? Were language barriers an issue in your experience?

One curiosity about assessment I would share is the balance between scalability and a good assessment. I've seen really great assessments that seem like they'd just be overwhelming to provide constructive feedback if you had more than 30-40 students (@Drew Boso mentioned questions about the time commitment in that vein!). Things like longer-term collaborative projects, oral exams, case studies, short answer / essays, etc. all seem great until you've got 150 of them to grade in a weekend. 

If you'll indulge me in sharing a brief bit about a current project, I work at Digital Promise and we're currently developing the Benchmark Assessment for Statistics Introductory Concepts (BASIC), a test designed to measure a student's ability to apply the key statistical concepts taught in a modern, postsecondary introductory statistics course. We presented on our progress thus far at JSM (Joint Statistical Meeting between MAA and ASA) and presented specifically on the balance between scalability and good assessment regarding Ethics items at SREE (Society for the Research on Educational Effectiveness). We've opted to go for an entirely multiple-choice assessment (scalability) and have been working incredibly hard to ensure it's still a good assessment (e.g., several pilot iterations, A/B testing, Pre-Test/Post-Test, cognitive interviews, IRT, DIF, etc.) It's certainly not easy, but I'm excited to get to a final version that we release to the field and to see others' answers to your questions about curiosities!

Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?