Difference between revisions of "Mastery Learning Templates"

From Open Pattern Repository for Online Learning Systems
Jump to navigation Jump to search
(Update link to author's page)
(Edited figure captions)
Line 77: Line 77:
[[File:Fig02 Mastery Learning Templates.png | center|600px]]
[[File:Fig02 Mastery Learning Templates.png | center|600px]]


<center>Fig. 2. Screenshot of an ASSISTments template used to generate corresponding hints for problem templates described in Figure 3 (Image courtesy of [[ASSISTments]]).</center>
<center>Fig. 2. Screenshot of an ASSISTments template used to generate corresponding hints for problem templates described in Figure 1 (Image courtesy of [[ASSISTments]]).</center>




[[File:Fig03 Mastery Learning Templates.png | center|600px]]
[[File:Fig03 Mastery Learning Templates.png | center|600px]]


<center>Fig. 3. Screenshot of an ASSISTments problem and its corresponding hints generated with the problem and hint templates shown in Figure 3 and Figure 4 (Image courtesy of [[ASSISTments]]).</center>
<center>Fig. 3. Screenshot of an ASSISTments problem and its corresponding hints generated with the problem and hint templates shown in Figure 1 and Figure 2 (Image courtesy of [[ASSISTments]]).</center>




Revision as of 08:48, 10 March 2017


Mastery Learning Templates
Mastery Learning Templates.png
Contributors Paul Salvador Inventado, Peter Scupelli
Last modification March 10, 2017
Source Inventado and Scupelli (2016).[1]
Pattern formats OPR Alexandrian
Usability
Learning domain
Stakeholders

Create templates that can be used to generate problems and corresponding hints for mastery learning exercises instead of encoding problem and hint variations manually.

Context

An online learning system for math is used to help students master a skill they recently learned. One way to implement mastery learning[2][3][4] is through problem-solving exercises that target a particular skill or set of skills. Students need to answer the exercises until they achieve a particular level of performance that equates to mastery (e.g., answer three problems correctly in a row, answer 90% of the problems correctly, answer all problems correctly within a given time limit). Another important aspect of mastery learning is learning support, such as hints, to help students remember key concepts, identify their mistakes, or make inferences[5]. Learning support may help students solve the current problem or other related problems and achieve mastery.

Problem

Encoding problem and learning support content for mastery learning exercises in a math online learning system is tedious and prone to errors.

Forces

  1. Problem variations. Problems designed to test the same skill are often very similar, but are tedious to replicate (e.g., Problem: Simplify 6/21; Answer: 2/7 vs. Problem: Simplify 9/24; Answer: 3/8 vs. Problem: Simplify 15/30; Answer: 1/2).
  2. Learning support variations. Learning support associated with problem variations are also similar and tedious to replicate (e.g., Hint: What number divides both 6 and 21? vs. Hint: What number divides both 9 and 24? vs. Hint: What number divides both 15 and 30?).
  3. Replication error. Content creators may use “shortcuts” to save time and effort to encode similar content into a system. For example, content is copied and pasted then edited to produce different variations. Creating subtle variations may be confusing and could lead to errors such as duplicate variations, missing variations, unsuspected deletion of content, or unsuspected addition of content.

Solution

Therefore, create templates that can be used generate problem and learning support variations. A template contains the content, but with variations replaced by placeholders. For example – Problem: Simplify %v{a}/%v{b}; Answer: %v{answer}; Hint: What number divides both %v{a} and %v{b}. In this particular example, %v{a} is a placeholder for variable a that is later populated by a content generator such as what is described in the Mastery Learning Exercise Generator design pattern (see Figures 1-3). Variables are merged with templates to produce problem variations. The example shown in the forces section for example, can be generated by configuring variable a to contain values that are multiples of three, greater than three, and less than or equal to 15 (i.e., 6, 9, 12, 15), and variable b to contain values that are multiples of three, greater than 20, and less than or equal to 30 (i.e., 21, 24, 27, 30). Answers can be stored in variables as well (i.e., 2/7, 3/8, 4/9, 1/2).

Consequences

Benefits

  1. Content creators only need to create problem templates and specify placeholder values instead of encoding each problem variation.
  2. Content creators can create learning support templates with associated placeholder values instead of encoding each variation.
  3. Encoding templates and identifying variable values separately require less cognitive load compared to encoding every variation. The reduction of cognitive load and frequency of encoding content may help content creators avoid mistakes[6][7]

Liabilities

  1. The online learning system will need to support templates and problem generation.
  2. Teachers and content creators need to learn how to use templates and other modules needed to automatically generate content.
  3. Teachers and content creators need to carefully select variable values to avoid unintended problem or learning support variations. For example, content creators need to ensure that zeroes are not generated in the denominator of fraction values.
  4. Creating a template is more difficult than encoding a problem. It is only worthwhile to create templates when there are many problem variations, which are tedious to encode and prone to errors.
  5. Individual differences and learning contexts may affect the effectiveness of math problems and their associated learning support[8][9][10][11]. Templates may need to be generated for specific student populations.

Evidence

Literature

  1. Mastery learning is a method of instruction that encourages student mastery of prerequisite knowledge before moving on to a subsequent topic[2]. It involves repeated learning support and testing until mastery of the topic is reached. Plenty of research has been conducted on mastery learning since it was introduced by Bloom. There have been debates about its effectiveness as a method of instruction, but most research findings still conclude that mastery learning has positive effects on student achievement and attitude (cf. Cox (1979)[12], Guskey (2008)[3], Guskey & Gates (1985)[13], Kulik et al. (1990)[4], Slavin (1987)[14]).
  2. Errors in man-machine systems may be attributed to system issues or to human error[7]. Errors may either be constant – average difference between actual outcomes and expected outcomes; or variable – variance between actual outcomes. Constant errors are often easier to correct because the system can be re-calibrated. Variable errors on the other hand require additional support to keep actions more consistent (e.g., training, immediate feedback, task simplification).
  3. Repetitive tasks, especially those that require more attention, lead to high cognitive load and increase the probability of commiting errors.[6]
  4. Processing similar items in working memory can cause confusion that may lead to decrease in performance.[15]

Data

Content in the ASSISTments online learning system is largely contributed by content creators and teachers[16]. ASSISTments has a commenting functionality that students and teachers can use to report issues with problems or hints in the system. According to an analysis conducted on the comments received, around 45% of the comments were helpful in addressing system issues[17]. Other comments were either repetitions or unrelated to ASSISTments. One common issue reported is that some problems, answers, and hints did not match or were encoded incorrectly. Errors might have been introduced when content creators encoded problem variations and got confused with the input values.


Related patterns

The Mastery Learning Templates design pattern is used by the Mastery Learning Exercise Generator design pattern to generate exercises for mastery learning. Multiple hints may be associated with a single problem variation so students can request for more hints on a problem as described in the Increasing Hint Specificity design pattern. The automatic generation of exercises implements the Try It Yourself design pattern, and generating multiple variations of a problem implements the One Concept Several Implementations design pattern[18].

Example

Templates are commonly used in learning systems for content generation. Systems may use different syntax for their templates and utilize different mechanisms for setting and assigning placeholder values, but it is still a preferred approach because of its simplicity. For example, Cognitive Tutor Algebra uses model tracing to analyze the correctness of each step in students’ solution as they solve an algebra problem[19]. An example of an incorrect step may be: failure to simplify an equation. The incorrect steps identified through model tracing are used to populate placeholder values in hint templates and to generate hints that are shown to students who request help. Various hints are generated with a template without requiring the content creator to make each one.

Intelligent Teaching Assistant for Programming (ITAP) helps students learn to program in Python by asking them to solve programming problems and providing them hints that describe what they can do next to reach the correct solution[20]. ITAP uses a data-driven approach, which utilizes teachers’ exemplar solutions and previously submitted student solutions to build a solution space. The most similar program state in the solution space is used to identify the next step that can be taken to transform the student’s solution into a correct solution. The change between the student’s solution and the next step is extracted and used to populate placeholders in a hint template to generate the hint presented to the student.

ASSISTments is an online learning system that teachers use to create problems with associated learning support[16][17]. It supports the creation of problem, answer, and learning support templates, which content creators use to generate problem variations. Variables are input into the interface so they can be merged with the template to generate the problem and hint presented to the student.

Figure 1 shows a screenshot of ASSISTments’ interface for creating problem templates. The problem uses placeholders for variables a, b, and answer, which are assigned the values specified in the Variables section. Figure 2 shows a hint template for the problem shown in Figure 1, which also uses variables a, b, and answer to generate the corresponding hints. Figure 3 shows an example of a problem and hint variation created using both templates.

Fig01 Mastery Learning Templates.png
Fig. 1. Screenshot of an ASSISTments template used to generate problem variations and their corresponding answers (Image courtesy of ASSISTments)


Fig02 Mastery Learning Templates.png
Fig. 2. Screenshot of an ASSISTments template used to generate corresponding hints for problem templates described in Figure 1 (Image courtesy of ASSISTments).


Fig03 Mastery Learning Templates.png
Fig. 3. Screenshot of an ASSISTments problem and its corresponding hints generated with the problem and hint templates shown in Figure 1 and Figure 2 (Image courtesy of ASSISTments).


References

  1. ↑ Inventado, P.S. & Scupelli, P. (2016). Design Patterns for Math Problems and Learning Support in Online Learning Systems. In Proceedings of the 11th Viking Conference on Pattern Languages of Programs, VikingPLoP 2016. ACM: New York, NY, USA.
  2. ↑ 2.0 2.1 Bloom, B. S. (1968). Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1. Evaluation comment, 1(2), n2.
  3. ↑ 3.0 3.1 Guskey, T.R. (2008). Mastery Learning. In Good, T.L. (Ed.), 21st Century Education: A Reference Handbook. Thousand Oak, CA: Sage Publicaitons, I-254.
  4. ↑ 4.0 4.1 Kulik, C. L. C., Kulik, J. A., & Bangert-Drowns, R. L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of educational research, 60(2), 265-299.
  5. ↑ Hume, G., Michael, J., Rovick, A., & Evens, M. (1996). Hinting as a tactic in one-on-one tutoring. The Journal of the Learning Sciences, 5(1), 23-47.
  6. ↑ 6.0 6.1 Head, J., & Helton, W. S. (2014). Sustained attention failures are primarily due to sustained cognitive load not task monotony. Acta psychologica, 153, 87-94.
  7. ↑ 7.0 7.1 Chapanis, A., Garner, W. R., & Morgan, C. T. (1949). Applied experimental psychology: Human factors in engineering design. New York, NY: John Wiley & Sons.
  8. ↑ Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.). Metacognition: Knowing about knowing. Cambridge, MA: MIT Press, 185-205.
  9. ↑ Cen, H., Koedinger, K. R., & Junker, B. (2007). Is Over Practice Necessary?-Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining. Frontiers in Artificial Intelligence and Applications, 158, 511.
  10. ↑ Inventado, P.S., Scupelli, P., VanInwegen, E., Ostrow, K., Heffernan, N., Ocumpaugh, J., Baker, R., Slater, S., & Almeda, V. (2016). Hint Availability Slows Completion Times in Summer Work. In Tiffany Barnes, Min Chi and Mingyu Feng (Eds.). In Proceedings of the 9th International Conference on Educational Data Mining.
  11. ↑ Salden, R. J., Aleven, V., Schwonke, R., & Renkl, A. (2010). The expertise reversal effect and worked examples in tutored problem solving. Instructional Science, 38(3), 289-307.
  12. ↑ Cox Jr, W. F., & Dunn, T. G. (1979). Mastery learning: A psychological trap?. Educational Psychologist, 14(1), 24-29.
  13. ↑ Guskey, T.R., & Gates, S.L. (1985). A Synthesis of Research on Group-Based Mastery Learning Programs. Paper presented at 69th the Annual Meeting of the American Educational Research Association, 66 pages.
  14. ↑ Slavin, R. E. (1987). Mastery learning reconsidered. Review of educational research, 57(2), 175-213.
  15. ↑ Wickens, C. D., Lee, J., Liu, Y., & Becker, S. G. (2004). An introduction to human factors engineering.
  16. ↑ 16.0 16.1 Heffernan, N.T. and Heffernan, C.L. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education 24, 4 (2014), 470-497.
  17. ↑ 17.0 17.1 Razzaq, L., Patvarczki, J., Almeida, S., Vartak, M., Feng, M., Heffernan, N. T., & Koedinger, K. R. (2008). The ASSISTment Builder: Supporting the life cycle of ITS content creation (WPI Tech Report No. WPI-CS-TR-08-06). Worcester, MA: Worcester Polytechnic Institute.
  18. ↑ Bergin, J., Eckstein, J., Völter, M., Sipos, M., Wallingford, E., Marquardt, K., Chandler, J., Sharp, H., and Manns, M.L. (2012). Pedagogical patterns: advice for educators. Joseph Bergin Software Tools.
  19. ↑ Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239-264.
  20. ↑ Rivers, K., & Koedinger, K. R. (2015). Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. International Journal of Artificial Intelligence in Education, 1-28.

External Links

Retrieved from "https://www.learningenvironmentslab.org/openpatternrepository/index.php?title=Mastery_Learning_Templates&oldid=1566"