IMPACT Live!

 View Only
  • 1.  Curiosity in Assessment

    Posted 10-06-2023 15:03:00

    Last semester, I finally decided to try a test format that I have always been curious about: oral tests. Overall, the oral tests were a very positive experience for both me and my students, and I am even trying them again this semester.  Now I have new curiosities about oral exams.  Curiosity has a way of working like this: as we reach one curiosity, another set of curiosities is revealed. Sometimes curiosity answers questions, sometimes curiosity brings up more questions. So, what curiosities about assessment would you want to share?



    ------------------------------
    Luke Walsh
    Instructor
    Catawba Valley CC
    Hickory NC
    ------------------------------


  • 2.  RE: Curiosity in Assessment

    Posted 10-07-2023 01:23:00

    Thanks for sharing, Luke! I've been on a journey over the past year to improve assessment in my courses too. Your interest in making it a better experience, perhaps even a learning experience, is really incredible. I'm curious about the logistics of your tests: how did you manage meeting with all of your students and how many standards or topics were you able to assess at a time? Also, could you share how you communicated progress to students? I'd love to know what your feedback looked like and whether or not students received that feedback in real time. 

    Thinking more broadly, I'm curious if, and in what ways, other instructors use reassessments in their courses. Luke wouldn't need to invent a new system! At a later time, students could test again with a new problem to demonstrate learning. I'm curious if more instructors are using grading as a way to increase learning in this way so that feedback loops become a positive experience for students. 



    ------------------------------
    Drew Boso
    Math Instructor
    Northwood Technical College
    Ashland WI
    ------------------------------



  • 3.  RE: Curiosity in Assessment

    Posted 10-07-2023 20:42:00
    Edited by Rick Powers 10-07-2023 20:49:39

    Greetings:

    I've been using oral assessments for a couple of years.  However, I prefer to call them "interviews" to avoid the connotations of "tests".

    The workload is OK for me, since I teach year-round with a reduced load.  It's important to note that interviews may not be suitable for every class (see below).  I rely on interviews for my math for liberal arts class, and I use them as part of the assessment of my statistics courses.

    Here are some of my observations:

    The interview environment may not be well-suited for every population of students.  Students on the lower levels often have more anxiety about interactions with the instructor, which can affect their performance for an interview.

    At the same time, interviews can be a useful option for those students who work better in speaking compared to writing.

    I've found that it is vital to address the emotional well-being of the student, and to guide them into a feeling of safety, and collaboration.  Part of that process is focusing on how students learn, e.g. having them bring their notebook of class notes and homework to the interview.  I emphasize students' ability to explain how to do a problem, so it's important to have those resources there.  This practice also allows me to provide guidance about better ways to take notes.

    I also provide a general set of questions before the interview, to help students identify what subjects they need to cover.  Typically, I cover five questions during a 30 minute session, and this is repeated four times per term.

    During the interview, if a student has not prepared at all, or hasn't brought their notebook, then we will set up another time -- at that point, they have a zero. If they are completely lost on a question, then they can choose to take a zero on that question, or we can work through it together.  If we run short of time, then we schedule another interview time. 

    I have perhaps a unique perspective on the assessment -- if I can help clear up a student's misunderstanding during the interview, then I will give them full credit for that question.  I rely on other types of assessment (online work, take-home tests) to provide a well-rounded skill assessment.  Since I offer opportunities to redo work on an exam, it makes sense that I can address concerns during the interview time.  

    It also takes some work for me to develop skill in interviewing.  Interviews can be especially challenging in an online class, doing video conferences, when I may not have established much of a relationship with the student.  One time, I made the mistake of insinuating that a student was cheating, since they were having trouble answering questions.  Not a good idea!  There are many reasons for a student having difficulty with responding, so it takes patience and skill to navigate that.

    In my opinion, the engagement and depth of understanding achieved through interviews is much richer than the standard testing environment. While it involves some risks, I believe the benefits outweigh the difficulties in most situations.



    ------------------------------
    Rick Powers
    Instructor
    Western Technical College
    La Crosse WI
    powersr@westerntc.edu
    ------------------------------



  • 4.  RE: Curiosity in Assessment

    Posted 10-19-2023 08:53:00
    • I like the renaming of an interview.  At the same time, I like the idea reclaiming what a test can be.  Maybe we join them together, "Interview Test" ;-)
    • All of your feedback and comments I love.  Your approach is very similar to mine.
    • Next semester I am teaching a stats course and I plan on trying it out for the first time.
    • Perhaps, if you have not done so already, share your wealth of experiences at the next AMATYC or a local conference. 


    ------------------------------
    Luke Walsh
    Instructor
    Catawba Valley CC
    Hickory NC
    ------------------------------



  • 5.  RE: Curiosity in Assessment

    Posted 10-19-2023 08:47:00
    1. how did you manage meeting with all of your students: My class size was 10-15 students and the time spent ranged from 20-60 minutes. Students would sign up for a time slot by seeing what times were free on my calendar.
    2. how many standards or topics were you able to assess at a time?:  Usually one big topic.  For example, in differential equations there were three randomly produced equations that covered different techniques in solving.  I started by saying, "Pick which one you want to solve first.", "Okay, #2, what type of equation is that?".  In Calc III, the big topic was "Vectors".
    3. could you share how you communicated progress to students? Students would write on a white board in the classroom. We did a lot of classroom thinking during the course, so the format of chatting about math and writing out thinking was familiar to them. As students, talked and wrote during the assessment I would usually give feedback about after each sentence or each step with phrases like, "Nice", "Oh, I like that", "Explain that more.", "I don't agree."  If students appeared stuck or took a long a pause, I would try to break the silence, "No rush. Can you share what is going through your mind?"
    4. ...become a positive experience for students.: The student comments did cover a wide spectrum like, "I am really nervous about this.", "I am not used to sharing so much out loud.", "I am excited and curious about this, I have never taken a test like this.".  But what made this test format different for me is that I shared these experiences with those students instead of them having a positive/negative experience by themselves.  That is what I valued most.


    ------------------------------
    Luke Walsh
    Instructor
    Catawba Valley CC
    Hickory NC
    ------------------------------



  • 6.  RE: Curiosity in Assessment

    Posted 10-09-2023 12:55:00

    Yes thank you for sharing, Luke! 

    Before answering your question, I have two questions of my own! :)

    My Questions - Were students given the opportunity to choose between an oral exam and a written exam? Were language barriers an issue in your experience?

    One curiosity about assessment I would share is the balance between scalability and a good assessment. I've seen really great assessments that seem like they'd just be overwhelming to provide constructive feedback if you had more than 30-40 students (@Drew Boso mentioned questions about the time commitment in that vein!). Things like longer-term collaborative projects, oral exams, case studies, short answer / essays, etc. all seem great until you've got 150 of them to grade in a weekend. 

    If you'll indulge me in sharing a brief bit about a current project, I work at Digital Promise and we're currently developing the Benchmark Assessment for Statistics Introductory Concepts (BASIC), a test designed to measure a student's ability to apply the key statistical concepts taught in a modern, postsecondary introductory statistics course. We presented on our progress thus far at JSM (Joint Statistical Meeting between MAA and ASA) and presented specifically on the balance between scalability and good assessment regarding Ethics items at SREE (Society for the Research on Educational Effectiveness). We've opted to go for an entirely multiple-choice assessment (scalability) and have been working incredibly hard to ensure it's still a good assessment (e.g., several pilot iterations, A/B testing, Pre-Test/Post-Test, cognitive interviews, IRT, DIF, etc.) It's certainly not easy, but I'm excited to get to a final version that we release to the field and to see others' answers to your questions about curiosities!



    ------------------------------
    Julie Neisler
    Director of Quantitative Researcher
    Digital Promise
    Washington DC
    ------------------------------



  • 7.  RE: Curiosity in Assessment

    Posted 10-19-2023 09:14:00
    1. Were students given the opportunity to choose between an oral exam and a written exam?: There was not a choice.  However, there were sections that were broken into two, and students could choose which part they would an oral test or a project.
    2. Were language barriers an issue in your experience? Mostly all of my students had English as the first language. I did have one student that didn't and struggled with reading comprehension with word problems.  The oral format seemed to help that student.  As for mathematics language, I would ask students to interpret vocabulary words that they would use, "You just said a homogenous equation, what does the mean?" or "You just took the dot product of two vectors what does that tell us? Is that a vector or a scaler?"  I sorta followed a script, but often as I was able to personalize the oral test based on the student's performance. 

    Congrats with your stats project.  Looking through, it certainly shows a lot of thought, effort, and time has been given to that project.  Thanks for pushing the boundaries of scalability and assessment.  One personal challenge with building an assessment is that the questions are designed for the average/typical student.  With that focus, it can be easy to overlook those students that deviate from the center. 

    Keep up the awesome work!



    ------------------------------
    Luke Walsh
    Instructor
    Catawba Valley CC
    Hickory NC
    ------------------------------