Reference Solution/alx

Reference Solution
Contributors	Joseph Bergin, Christian Kohls, Christian Köppe, Yishay Mor, Michel Portier, Till Schümmer, Steven Warburton
Last modification	June 6, 2017
Source	Bergin et al. (in press 2015); Warburton et al. (2016)
Pattern formats	OPR Alexandrian
Usability
Learning domain
Stakeholders

Use one potential solution as a reference to relate scores to (partly) correct solutions.

You are planning how to mark an upcoming student assignment or task. You expect a correct answer, but realize there are Multiple Right Ways (Multiple Right Ways).

***

A student solution can be technically wrong, but the student may still be on the right track. A binary measure does not account for the actual performance.

A student might start out on a different path to a solution than you expected. It might prove fruitful, so simple measures don’t account for creativity.

If there is only one standard solution it is easy to score the performance by simply considering which parts of the standard solution are fulfilled. However, if you see only parts of an alternative solution it is hard to tell whether this solution is correct.

Even if the parts of a solution are correct, the parts likely require different amounts of effort and/or creativity.

Some of the key parts of the solution might be correct, but others less so.

All correct solutions score hundred percent but you have to find scores for partly correct solutions as well. Deciding what is partly correct becomes very difficult if a student uses an unexpected, yet potentially correct, solution approach.

***

Therefore, write one reference solution and identify the core functional elements of the solution. Assign scores to each solution part. When a student has given a partial solution then you can provide the scores for each solution part that has been found. It does not have to be the same solution as you use in your reference.

This is a divide and conquer approach to identify solution parts. The reference solution helps you to identify which essential parts are needed and you can also evaluate the difficulty of the approach, e.g. do students have to apply standard mechanisms (e.g. a for loop), apply a trick, have to understand a new model or develop a new solution. Both the essential parts needed and the standard mechanisms students have to apply should have direct connections with the Learning Outcomes (Learning Outcomes). These Learning Outcomes (Learning Outcomes) are therefore also leading in the process of writing the reference solution, which in consequence ensures Constructive Alignment (Constructive Alignment).

The reference solution does not set up the one right answer but can be used as a reference to map scores to partial correct solutions.

It can also help to identify requirements or steps that are required in any potential solution approach. If you have already several solution alternatives in mind you may write down several reference solutions and map the scores between them as well.

You may expect: Scoring becomes easier as you do not have to decide each time whether some solution fragment scores or not. If you identify a solution fragment you can easily assign a score based on the reference solution.

This leads to fairer scoring and it makes it easier to explain the scores to students and provide feedback. You can also publish a reference solution for a Mock Exam (Mock Exam) to demonstrate to students how you assign scores. You can also publish the reference solution after a written text to help students check how they performed right after the test - but make sure to communicate that it is not the only right solution.

This approach helps to map solution parts to a score if these parts are easily identified as correct.

However, if some part cannot be mapped to the reference solution this does not necessarily mean that it is wrong. Rather it needs to be evaluated for its correctness and a score needs to be found.

Related patterns

—is based on the Assessment Criteria List (Assessment Criteria List)

—can be part of a Mock Exam (Mock Exam) for allowing students’ Self Assessment (Self Assessment)

—should take Multiple Right Ways (Multiple Right Ways) into account

—helps with realizing Constructive Alignment (Constructive Alignment) by focusing on the Learning Outcomes (Learning Outcomes)

Examples

—A reference solution may store data in an array. Identify the need to store data somewhere can be assigned 2 points. If a student finds any alternative ways to store the data this counts 2 points.

—The reference solution identifies that at some point a new thread needs to be created. The correct syntax of creating the thread is scored one point. If a student has a different solution approach (correct or even incorrect) the creation of a thread at some point scores one point.

—A reference solution may need to fulfill some requirements. The reference solution shows that these requirements can actually be fulfilled. However, all other solution that fulfill the requirements score equally.