A Legacy of W.K. Kellogg

Assessment Steps for Every Level of Assessment[6]

Learning Outcomes

  1. Ensure that your Learning Outcomes are clear and effective:

    Clearly articulate the knowledge, skills, and attitudes that learners should have at the end of your course. Make your learning outcomes clear and understandable to people who are not familiar with your course. Anyone reading the learning outcomes should basically agree on what the learning outcomes mean.

    Example of less clear learning outcome: Students will appreciate the value of citations in academic work.

    Example of clearer learning outcome: Students will use complete, appropriately formatted citations in their papers and presentations. Students will be able to articulate at least three specific reasons why proper citations are valued when communicating in writing.

  2. Determine specific assessment data will be used that will reflect your learning outcomes. Data can be either quantitative or qualitative.

    There are two basic kinds of assessment evidence:

    • Direct Evidence: Specific learning artifacts generated by students, such as test problems, responses to assignments, performances, and so forth.

      Example: Your learning outcome may stipulate that students use complete, appropriately formatted citations in their papers and presentations. Comparing the difference in progression between their first and final draft would be direct evidence of learning.

    • Indirect evidence: Students’ reflections or opinions on their learning, such as surveys asking if they thought a particular activity was helpful.

      Example: Asking students to write a short reflection on how the research paper helped them to learn to write citations would provide indirect evidence.

Why Don't Final Grades Work for Assessment?

Grades can be used as data points in assessment, but grade data usually requires modification for assessment use.

The two problems with grades are:

  1. Despite our best efforts, graded assignments are often not well aligned with learning outcomes. For example: consider a course in which the learning outcome is critical thinking, but grades are generated from tests that mostly address simple content memorization.
  2. Grades are not pure measurements of learning outcomes achievement. For example, consider a paper: part of the grade is almost certainly based on timeliness. A student whose paper demonstrated excellent mastery of content and skill learning outcomes might have lost points for lateness.

How would we fix these situations so that it does not feel like we did all that work grading, only to lose everything when it comes to seeing if the course works?

  1. Pick out the test questions that address critical thinking and assess the results from those questions. If no test questions address critical thinking, that's a good indication that some sort of change is required.
  2. Choose relevant parts of the grading rubric for assessment use, rather than relying on the final grade for the paper.
  3. Decide on reasonable-but-challenging criteria for "success." What percent of students must achieve XYZ, at what level, in order for you to have a robust claim that the students have met your Learning Outcome?

    Example: The learning outcome that states students will use complete, appropriately formatted citations in their papers and presentations. You must determine how many students must achieve some level of correctness for you to be satisfied that your class met the learning outcome. Should it be that 80% of the students have 90% of their citations correct? Should it be that 50% of the students have 50% of the citations correct? This is really up to you, but it is important to consider what a reasonable number may be. (100% of the students, 100% of the time, is generally not reasonable.)

  4. Systematically gather and organize the evidence. It's better to gather less data, carefully selected, than to gather a lot of data without a plan. Not every piece of student work or even every piece of graded work has to count as assessment data.

    Analyzing Your Data (the first entry offers sample answers)
    Assessing What? Assessment Method Type of Data Collected What Do the Results Say? What Will You Do? Any Surprises?
    Student knowledge of course content prior to beginning course Student background probe Quantitative survey data and open-ended response 60% of students have some pre-knowledge. 40% have none. Challenge those who know, bring those who don't up to speed I thought more students would already know the information I asked for.
               
               

    Course Based Review and Assessment: Methods for Understanding Student Learning by Office of Academic Planning & Assessment, University of Massachusetts Amherst

  5. Decide if a change is needed to help students reach a higher level of success, based on your pre-established criteria. Make a plan to implement the change.

    Here is a helpful Action Plan Worksheet from UMass Amherst (entries 1 and 2 offer sample answers):
    Action Items Action of Take Steps to Implement
    1 Encourage students who have demonstrated prior knowledge
    1. From the data, identify content areas of greater knowledge
    2. Develop extra credit tasks for students to complete to build on their knowledge
    3. Establish a reward system to eliminate sense of extra work as "punitive"
    2 Bring those with less prior knowledge up to speed
    1.  
    2.  
    3.  
    3  
    1.  
    2.  
    3.  
    4  
    1.  
    2.  
    3.  
    5  
    1.  
    2.  
    3.  

    Course Based Review and Assessment: Methods for Understanding Student Learning by Office of Academic Planning & Assessment, University of Massachusetts Amherst