Thanks for sharing, Luke! I've been on a journey over the past year to improve assessment in my courses too. Your interest in making it a better experience, perhaps even a learning experience, is really incredible. I'm curious about the logistics of your tests: how did you manage meeting with all of your students and how many standards or topics were you able to assess at a time? Also, could you share how you communicated progress to students? I'd love to know what your feedback looked like and whether or not students received that feedback in real time.
Thinking more broadly, I'm curious if, and in what ways, other instructors use reassessments in their courses. Luke wouldn't need to invent a new system! At a later time, students could test again with a new problem to demonstrate learning. I'm curious if more instructors are using grading as a way to increase learning in this way so that feedback loops become a positive experience for students.
I've been using oral assessments for a couple of years. However, I prefer to call them "interviews" to avoid the connotations of "tests".
The workload is OK for me, since I teach year-round with a reduced load. It's important to note that interviews may not be suitable for every class (see below). I rely on interviews for my math for liberal arts class, and I use them as part of the assessment of my statistics courses.
Here are some of my observations:
The interview environment may not be well-suited for every population of students. Students on the lower levels often have more anxiety about interactions with the instructor, which can affect their performance for an interview.
At the same time, interviews can be a useful option for those students who work better in speaking compared to writing.
I've found that it is vital to address the emotional well-being of the student, and to guide them into a feeling of safety, and collaboration. Part of that process is focusing on how students learn, e.g. having them bring their notebook of class notes and homework to the interview. I emphasize students' ability to explain how to do a problem, so it's important to have those resources there. This practice also allows me to provide guidance about better ways to take notes.
I also provide a general set of questions before the interview, to help students identify what subjects they need to cover. Typically, I cover five questions during a 30 minute session, and this is repeated four times per term.
During the interview, if a student has not prepared at all, or hasn't brought their notebook, then we will set up another time -- at that point, they have a zero. If they are completely lost on a question, then they can choose to take a zero on that question, or we can work through it together. If we run short of time, then we schedule another interview time.
I have perhaps a unique perspective on the assessment -- if I can help clear up a student's misunderstanding during the interview, then I will give them full credit for that question. I rely on other types of assessment (online work, take-home tests) to provide a well-rounded skill assessment. Since I offer opportunities to redo work on an exam, it makes sense that I can address concerns during the interview time.
It also takes some work for me to develop skill in interviewing. Interviews can be especially challenging in an online class, doing video conferences, when I may not have established much of a relationship with the student. One time, I made the mistake of insinuating that a student was cheating, since they were having trouble answering questions. Not a good idea! There are many reasons for a student having difficulty with responding, so it takes patience and skill to navigate that.
In my opinion, the engagement and depth of understanding achieved through interviews is much richer than the standard testing environment. While it involves some risks, I believe the benefits outweigh the difficulties in most situations.
------------------------------Rick PowersInstructorWestern Technical CollegeLa Crosse WIpowersr@westerntc.edu------------------------------
------------------------------Drew BosoMath InstructorNorthwood Technical CollegeAshland WIOriginal Message:Sent: 10-06-2023 15:03:06From: Luke WalshSubject: Curiosity in Assessment
------------------------------Luke WalshInstructorCatawba Valley CCHickory NC------------------------------
------------------------------Rick PowersInstructorWestern Technical CollegeLa Crosse WIpowersr@westerntc.eduOriginal Message:Sent: 10-07-2023 01:22:44From: Drew BosoSubject: Curiosity in Assessment
Yes thank you for sharing, Luke!
Before answering your question, I have two questions of my own! :)
My Questions - Were students given the opportunity to choose between an oral exam and a written exam? Were language barriers an issue in your experience?
One curiosity about assessment I would share is the balance between scalability and a good assessment. I've seen really great assessments that seem like they'd just be overwhelming to provide constructive feedback if you had more than 30-40 students (@Drew Boso mentioned questions about the time commitment in that vein!). Things like longer-term collaborative projects, oral exams, case studies, short answer / essays, etc. all seem great until you've got 150 of them to grade in a weekend.
If you'll indulge me in sharing a brief bit about a current project, I work at Digital Promise and we're currently developing the Benchmark Assessment for Statistics Introductory Concepts (BASIC), a test designed to measure a student's ability to apply the key statistical concepts taught in a modern, postsecondary introductory statistics course. We presented on our progress thus far at JSM (Joint Statistical Meeting between MAA and ASA) and presented specifically on the balance between scalability and good assessment regarding Ethics items at SREE (Society for the Research on Educational Effectiveness). We've opted to go for an entirely multiple-choice assessment (scalability) and have been working incredibly hard to ensure it's still a good assessment (e.g., several pilot iterations, A/B testing, Pre-Test/Post-Test, cognitive interviews, IRT, DIF, etc.) It's certainly not easy, but I'm excited to get to a final version that we release to the field and to see others' answers to your questions about curiosities!
Congrats with your stats project. Looking through, it certainly shows a lot of thought, effort, and time has been given to that project. Thanks for pushing the boundaries of scalability and assessment. One personal challenge with building an assessment is that the questions are designed for the average/typical student. With that focus, it can be easy to overlook those students that deviate from the center.
Keep up the awesome work!
AMATYC Headquarters, Southwest Tennessee Community College5983 Macon CoveMemphis, TN 38134
Phone1-901-333-5643Email - firstname.lastname@example.orgEmail - myAMATYC/Technical email@example.com
Join AMATYCBenefits of MembershipLearn more about AMATYC