Blended Evaluation/OG
| Blended Evaluation | |
| Contributors | Oswald Comber |
|---|---|
| Last modification | May 12, 2017 |
| Source | Comber (2014)[1] |
| Pattern formats | OPR Alexandrian |
| Usability | |
| Learning domain | |
| Stakeholders | |
Blended evaluation is employed to consider a mix of self evaluation, peer evaluation and instructor evaluation in the assessment phases of a course. While the process is more demanding for all involved, it caters for increased student involvement and transparency of the evaluation and grading process.
Problem
Assigning grades to student contributions is the classic way of doing assessment in courses, but the benefit for students and the quality of information conveyed are typically low. An alternative – or complement – to classical grading is to provide feedback, which offers more meaningful and useful information than a simple grade or score. Still, that information is usually provided by one single person, i.e. the instructor. An approach to widen the horizon of perspectives is blended evaluation, which includes learners, instructors and peers as sources of feedback.
Context
Blended evaluation is applicable in any educational context in which instructors are willing to consider multiple perspectives in assessment, and where learners are capable of providing constructive feedback and a critical view on their own contributions. Blended evaluation means a combination of different evaluation types and it is also blended in the sense of ‘blended learning’ (as Wallace[2] defines blended learning) – namely a combination of face-to-face (f2f) interaction and e-learning. In this context it is important that f2f-interactions and e-learning-elements are not simply put together in a course, but that f2f-interactions and e-learning-elements are aligned and that they stand in a relationship of mutual exchange. Online tools help to reduce the effort of collecting assignments and delivering feedback in all evaluation tasks, particularly in the peer-evaluation component.
Solution
Blended evaluation combines the views of different persons, whereas in contrast blended assessment combines different assessment methods (e.g. multiple choice tests, project works, or presentations). The blended evaluation pattern is composed of the following processes: – self evaluation – peer evaluation, and – instructor evaluation.
Instead of traditional grading, learners receive feedback in the form of blended evaluation. Blended evaluation means a combination of self evaluation, peer evaluation and instructor evaluation, which is used to provide more detailed and multifaceted feedback than grading. Note that blended evaluation is used to evaluate one set of student contributions from multiple perspectives, for instance a project is evaluated by the project owner, by peers working on similar projects, and by the instructor. To provide for transparency in the process, evaluations can be made available electronically to participating students.
To achieve a shared understanding we start with a brief summary of a few basic terms. Evaluation means the process of giving feedback by tutors, co students and lecturers to an evaluee and rating an evaluation object. The evaluation object can be any contribution, process, or accomplishment. The person who contributes the object of evaluation is the evaluee. The evaluator is the one who is doing the evaluation of the object. In the case of self evaluation this means that the creator of an evaluation object and the evaluator are the same person. The communication between evaluee and evaluator follows a simple structure (Figure 1).
Evaluations can either be unidirectional (and in the case of peer evaluation can be anonymous) or can allow the evaluees to communicate with the evaluators, for asking questions, clarifying or discussing certain aspects of their work.
A typical course
A typical course that used blended evaluation might take place in the following way. The course starts with an initial face-to-face meeting, in which students form teams and either are assigned to tasks or allowed to choose their tasks by themselves. Then the student teams work on their assignment and submit their results onto an e-learning system for peer evaluation. The peer evaluation is performed online by peers from other teams. After the peer evaluation, the various outcomes of the assignments are improved according to the peer feedback. In the next step a presentation of the assignments follows. Each student has to perform a self evaluation of his/her own contribution. The self evaluation provides first-hand reflections and information about the activities and efforts of individual students. Alternatively each team can work collaboratively on their team evaluation. This variant is suggested when the team is allowed to make a direct grade suggestion for each team member. Finally there is an evaluation by the instructor. The grade for the course is assessed by the instructor based on all the evaluations.
Self evaluation
In the classic approach an important perspective on the learning process is often neglected, namely the perspective of the learner himself or herself. Providing an opportunity for a self evaluation accommodates this perspective. In a self evaluation process, students evaluate their achievements and contributions. This can be achieved by requesting either free text evaluations or by the use of structured questionnaires. The flow of activities of self evaluation is depicted in Figure 2.
In the preparation step, the objects to be evaluated and the evaluation criteria are defined. Usually this is done by the instructor, but it is also feasible to involve students in making these decisions. The next important step is to publish the self evaluation criteria and available guidelines, to ensure a transparent and well guided evaluation process, and then the learners perform the self evaluations. In the end the evaluations are provided to the lecturer(s) and/or other participants.
Peer evaluation
In peer evaluation scenarios students receive feedback from their peers, which encourages active participation and engagement of learners in the assessment process. Also students experience the task of peer reviewing and giving constructive feedback. The activities are depicted in Figure 3.
Note that it is possible to either assign learners to particular objects of evaluation or the learners may be allowed to choose freely. In any case it is important to provide for a fair distribution of evaluation objects, and to ensure that every object is evaluated at least once.
Instructor evaluation
Instructor evaluation aims at providing learners with feedback from the instructor as a complement to grading (Figure 4).
The first step is preparing the evaluation and the evaluation criteria. The publishing of evaluation criteria in the next step contributes to transparency in the process. Then the instructor performs the evaluation and provides the feedback to the learners. This can be done orally or in written form, e.g. via the e-learning system. Publishing the evaluation results is optional but it might be interesting for other learners to see and compare, so this could be discussed with students in the preparation of the evaluation criteria and process.
Support
Source
The source for the pattern Blended Evaluation (Blended Evaluation) is the design narrative ‘Web Engineering’.[3]
Theoretical justification
Traditional grading lacks information value for the learners and fairness, especially fairness towards learners that are struggling with their studies[4]. This process of grading cannot satisfy the standards of reliability, validity and objectivity[5]. Objectivity means that a grade is impartial and independent from the person who gives the grade, which in practice is not the case, as different teachers are found to give different grades for the same work. Reliability is achieved when the same teacher is able to arrive at the same grade at any time, but many teachers (and other evaluators) are not able to reproduce grades in a reliable way. Validity implies that a grade stands for a definable aspect (‘the learning’) that was measured, but learning is multidimensional (e.g. the personal and overall achievements, the efforts, the development during the process, the unrecognized contributions, the deeper understanding of topics), so valid measurement is virtually impossible.
Blended evaluation attempts to reduce these problematic aspects of grading by taking different perspectives and opinions on achievements and contributions into account, specifically the view of the learner, the view of the learner’s peers, and the view of the instructor. Rogers & Freiberg[6] state that the primary evaluator of learning is the learner, influenced and enriched by the group and a facilitator. In blended evaluation all three aspects are incorporated.
Blended evaluation was designed in the context of learner-centered education, which strives to achieve a particular kind of prizing, acceptance and trust towards students as one major part of providing conditions for significant, beneficial learning[7]. Self evaluation, in particular, accentuates and cultivates the responsibility of the learner towards his or her (self-initiated) learning practice[8]. Learning is more successful when the interest of the learners rises. This can be fostered by encouraging learners to learn in a selfregulated way[9][10]. Blended evaluation contributes to enhancing the learning process, by enhancing the grading process with increased transparency, fairness, and student-inclusiveness.
References
- ↑ Comber, O. (2014). Pattern: Blended Evaluation. In Mor, Y., Mellar, H., Warburton, S., & Winters, N. (Eds.). Practical design patterns for teaching and learning with technology (pp. 293-299). Rotterdam, The Netherlands: Sense Publishers.
- ↑ Wallace, S. (Ed.). (2009). A dictionary of education. Retrieved July 21, 2009, from http://www.oxfordreference.com/views/BOOK_SEARCH.html?book=t267
- ↑ Comber, O. & Derntl, M. (2014). Design Narrative: Web Engineering. In Mor, Y., Mellar, H., Warburton, S., & Winters, N. (Eds.). Practical design patterns for teaching and learning with technology (pp. 245-249). Rotterdam, The Netherlands: Sense Publishers.
- ↑ Vierlinger, R. (1998). Plädoyer für die Abschaffung der Ziffernnoten. Erziehung heute (3). Retrieved July 21, 2009, from http://paedpsych.jk.uni-linz.ac.at/INTERNET/ORGANISATIONORD/VIERLINGERORD/VierlingerAbschaffung.html.
- ↑ Vierlinger, R. (1999). Leistung spricht für sich selbst. ‘Direkte Leistungsvorlage’ (Portfolios) statt Ziffernzensuren und Notenfetischismus. Heinsberg: Dieck-Verlah.
- ↑ Rogers, C. R., & Freiberg, H. J. (1994). Freedom to learn (3rd ed.). New York. Toronto: Merrill.
- ↑ Rogers, C. R. (1983). Freedom to learn for the 80’s (p.123). Columbus, Ohio: Charles E. Merrill Publishing Company.
- ↑ Rogers, C. R. (1983). Freedom to learn for the 80’s (pp. 158-159). Columbus, Ohio: Charles E. Merrill Publishing Company.
- ↑ Rogers, C. R. (1983). Freedom to learn for the 80’s. Columbus, Ohio: Charles E. Merrill Publishing Company.
- ↑ Stern, T. (2008). Förderliche Leistungsbewertung (p.21). Salzburg: Özeps.