Difference between revisions of "Explain Worked Solutions"

From Open Pattern Repository for Online Learning Systems
Jump to navigation Jump to search
m (Edited format)
(Added category)
Line 5: Line 5:
|image= Explain_Worked_Solutions.png
|image= Explain_Worked_Solutions.png
|contributor= [[Paul Salvador Inventado]], [[Peter Scupelli]]
|contributor= [[Paul Salvador Inventado]], [[Peter Scupelli]]
|source= Inventado and Scupelli (2016).<ref name="inventado">Inventado, P.S. & Scupelli, P. (2016). [http://dl.acm.org/authorize?N22598 Design Patterns for Math Problems and Learning Support in Online Learning Systems.] In ''Proceedings of the 11th Viking Conference on Pattern Languages of Programs, VikingPLoP 2016.'' ACM: New York, NY, USA.</ref>
|source= Inventado and Scupelli (2016).<ref name="inventado">Inventado, P.S. & Scupelli, P. (2016). [http://dl.acm.org/authorize?N22598 Design Patterns for Math Problems and Learning Support in Online Learning Systems.] In ''Proceedings of the 11th Viking Conference on Pattern Languages of Programs (VikingPLoP 2016).'' ACM: New York, NY, USA.</ref>
|dataanalysis= <!-- If applicable, list of data analyses used for mining the pattern separated by a " , "comma -->
|dataanalysis= <!-- If applicable, list of data analyses used for mining the pattern separated by a " , "comma -->
|domain= <!-- Learning domain the design pattern belongs to (e.g., General, Math, Algebra) -->
|domain= <!-- Learning domain the design pattern belongs to (e.g., General, Math, Algebra) -->
Line 56: Line 56:


===Data===
===Data===
Data was collected from the [[ASSISTments]] online learning system between September 2012 and September 2013. One of the features in the data set was students’ average frustration, which described how likely students experienced frustration while answering a problem in [[ASSISTments]]. Student frustration was predicted by an affect detector using features like the number of previous incorrect answers, time taken to solve problems, and number of hint requests<ref name="Ocumpaugh">Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). [http://onlinelibrary.wiley.com/doi/10.1111/bjet.12156/abstract Population validity for Educational Data Mining models: A case study in affect detection.] ''British Journal of Educational Technology, 45''(3), 487-501.</ref>. An analysis of frustration instances in the ASSISTments data set showed that some problems were more challenging than others based on the small number of students who answered it correctly. The analysis also revealed an interesting behavior wherein some students continually requested for hints until it showed them the solution and the correct answer, paused for a moment, submitted the correct answer, then moved on to the next problem. The next time these students encountered a similar problem, they answered it correctly on the first attempt. Students’ usage of hints, when taken together, resembled a worked solution. This may indicate students’ preference to learn from worked solutions, and its positive effects on their learning. More details about the data used, the methodology used for analysis, and the results are presented in<ref name="Inventado2015a">Inventado, P. S., & Scupelli, P. (2015). [http://dl.acm.org/citation.cfm?id=2855336 Data-driven design pattern production: a case study on the ASSISTments online learning system.] In ''Proceedings of the 20th European Conference on Pattern Languages of Programs, EuroPLoP 2015'' (p. 14). ACM:New York.</ref>.
Data was collected from the [[ASSISTments]] online learning system between September 2012 and September 2013. One of the features in the data set was students’ average frustration, which described how likely students experienced frustration while answering a problem in [[ASSISTments]]. Student frustration was predicted by an affect detector using features like the number of previous incorrect answers, time taken to solve problems, and number of hint requests<ref name="Ocumpaugh">Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). [http://onlinelibrary.wiley.com/doi/10.1111/bjet.12156/abstract Population validity for Educational Data Mining models: A case study in affect detection.] ''British Journal of Educational Technology, 45''(3), 487-501.</ref>. An analysis of frustration instances in the ASSISTments data set showed that some problems were more challenging than others based on the small number of students who answered it correctly. The analysis also revealed an interesting behavior wherein some students continually requested for hints until it showed them the solution and the correct answer, paused for a moment, submitted the correct answer, then moved on to the next problem. The next time these students encountered a similar problem, they answered it correctly on the first attempt. Students’ usage of hints, when taken together, resembled a worked solution. This may indicate students’ preference to learn from worked solutions, and its positive effects on their learning. More details about the data used, the methodology used for analysis, and the results are presented in<ref name="Inventado2015a">Inventado, P. S., & Scupelli, P. (2015). [http://dl.acm.org/citation.cfm?id=2855336 Data-driven design pattern production: a case study on the ASSISTments online learning system.] In ''Proceedings of the 20th European Conference on Pattern Languages of Programs (EuroPLoP 2015)'' (p. 14). ACM:New York.</ref>.




==Related patterns==
==Related patterns==
The {{Patternlink|Explain Worked Solutions}} design pattern closely resembles the {{Patternlink|Worked Examples}} design pattern<ref name="Inventado2015b">Inventado, P. S., & Scupelli, P. (2015). [https://cmu.box.com/shared/static/m6qfs01z71gt38a7tf85gcgl8t84iw50.pdf A Data-driven Methodology for Producing Online Learning System Design Patterns.] In ''Proceedings of the 22nd Pattern Languages of Programs conference, PLoP 2015'' (Vol. 15). ACM: New York.</ref>, but provides the answer to the current problem solved by the student. The {{Patternlink|Explain Worked Solutions}} design pattern can be used with the {{Patternlink|Try It Yourself}} design pattern<ref name="Bergin2012"/> to provide students with learning support while they answer problem-solving activities that promote better understanding of a given topic. Encoding worked solutions for problem variations may be tedious to write. Content creators may use the {{Patternlink|Mastery Learning Exercise Generator}} and {{Patternlink|Mastery Learning Templates}} design patterns to generate worked solutions automatically. It may be good idea to delay the presentation of worked solutions until students have attempted to solve the problem or tried to request basic hints as described in the {{Patternlink|Increasing Hint Specificity}} design pattern. The {{Patternlink|Image-Enhanced Hint}} design pattern<ref name="Inventado 2016a">Inventado, P.S. and Scupelli, P. (2016). [http://dl.acm.org/authorize.cfm?key=N21354 Design Patterns for Helping Students to Learn to Visualize Math Problems in Online Learning Systems.] In ''Proceedings of the 21st European Conference on Pattern Languages of Programs, EuroPLoP 2016''.</ref>. can be used to clarify worked solutions. If applicable, remember to implement the {{Patternlink|One Concept Several Implementations}} design pattern<ref name="Bergin2012"/> to make students aware of other solutions that may solve the problem.
The {{Patternlink|Explain Worked Solutions}} design pattern closely resembles the {{Patternlink|Worked Examples}} design pattern<ref name="Inventado2015b">Inventado, P. S., & Scupelli, P. (2015). [https://cmu.box.com/shared/static/m6qfs01z71gt38a7tf85gcgl8t84iw50.pdf A Data-driven Methodology for Producing Online Learning System Design Patterns.] In ''Proceedings of the 22nd Pattern Languages of Programs conference (PLoP 2015)'' (Vol. 15). ACM: New York.</ref>, but provides the answer to the current problem solved by the student. The {{Patternlink|Explain Worked Solutions}} design pattern can be used with the {{Patternlink|Try It Yourself}} design pattern<ref name="Bergin2012"/> to provide students with learning support while they answer problem-solving activities that promote better understanding of a given topic. Encoding worked solutions for problem variations may be tedious to write. Content creators may use the {{Patternlink|Mastery Learning Exercise Generator}} and {{Patternlink|Mastery Learning Templates}} design patterns to generate worked solutions automatically. It may be good idea to delay the presentation of worked solutions until students have attempted to solve the problem or tried to request basic hints as described in the {{Patternlink|Increasing Hint Specificity}} design pattern. The {{Patternlink|Image-Enhanced Hint}} design pattern<ref name="Inventado 2016a">Inventado, P.S. and Scupelli, P. (2016). [http://dl.acm.org/authorize.cfm?key=N21354 Design Patterns for Helping Students to Learn to Visualize Math Problems in Online Learning Systems.] In ''Proceedings of the 21st European Conference on Pattern Languages of Programs (EuroPLoP 2016)''.</ref>. can be used to clarify worked solutions. If applicable, remember to implement the {{Patternlink|One Concept Several Implementations}} design pattern<ref name="Bergin2012"/> to make students aware of other solutions that may solve the problem.




Line 81: Line 81:




[[Category:Design_patterns]] [[Category:ASSISTments]][[Category:Full_Pattern]]
[[Category:Design_patterns]] [[Category:ASSISTments]][[Category:Full_Pattern]]  [[Category:Pattern Language for Math problems and Learning Support in Online Learning Systems]] [[Category:Online Learning System]] [[Category:Intelligent Tutoring System]]

Revision as of 10:13, 15 May 2017


Explain Worked Solutions
Explain Worked Solutions.png
Contributors Paul Salvador Inventado, Peter Scupelli
Last modification May 15, 2017
Source Inventado and Scupelli (2016).[1]
Pattern formats OPR Alexandrian
Usability
Learning domain
Stakeholders

Provide students with clearly explained worked solutions when they are unable to answer problems correctly despite receiving support.

Context

Students are asked to answer problems in a math online learning system to practice a particular skill. The system provides students with different types of learning support to help them solve the problem such as hints generated using the Mastery Learning Templates and Mastery Learning Exercise Generator design patterns or guide questions built with the Scaffold Problems with Guide Questions design pattern.

Problem

Students may be unable to solve a problem even after making answer attempts and receiving learning support.

Forces

  1. Prior knowledge. Students cannot solve a problem if they lack the knowledge or skills necessary to solve it[2].
  2. Learning opportunity. Students lose a learning opportunity if they are not taught how to solve a problem they failed to answer.
  3. Disengagement. Students may disengage from the learning activity when they get stuck too long trying to solve a problem[3].

Solution

Therefore, allow students to request for worked solutions if they tried to answer a problem and requested help, but were still unable to solve it. The worked solution should show each step in the solution process that starts with the given problem until it is solved. Each step should be explained clearly so students understand how the result of one step relates to the next step. Students may have a better chance of understanding the process and applying it to similar problems if they understand the process. Students may learn more from a worked solution because: (a) they are familiar with the problem (as they already tried to solve it), (b) they may be able to figure out what step they did incorrectly or did not perform, and (c) concise worked solutions guide student learning and do not introduce extraneous information that may cause additional cognitive load[4][5]. Students may feel a sense of accomplishment when they recognize the steps they performed in the worked solution. There may be more than one way to solve a problem, but it may be a good idea to present only one worked solution at a time to avoid confusion. Consider using the One Concept Several Implementations design pattern[6] by presenting different worked solutions for different variations of a problem so students become aware of different ways to solve it.

Consequences

Benefits

  1. Worked solutions may help students acquire or relearn knowledge, which they can organize and store in long-term memory for later retrieval[7][2].
  2. Providing clear explanations in the worked solution will help learners understand and apply what they learned to solve similar problems in the future[8][9]
  3. Worked solutions may help students get unstuck from the current problem. Understanding the solution may also help them avoid getting stuck on similar problems in the future.

Liabilities

  1. Students may overlook critical information in the worked solution unless they are highlighted[8]
  2. Content creators will need to create a worked solution for each problem they create.
  3. Students may miss a learning opportunity when they ask for worked solutions too soon because the worked solution gives away the answer.
  4. Students might not learn how to construct solutions on their own if they are used to having worked solutions.
  5. Students may get the impression that a particular worked solution is the only way to solve the problem unless otherwise specified (e.g., explicitly stating there are other solutions, showing other solutions in other problems, asking students to compare their answers with their peers who might have used a different solution).

Evidence

Literature

  1. Learners need to recognize and understand the solution before they are able to apply it[8]
  2. Several studies were conducted to compare the effects of various feedback types on learning such as no feedback, knowledge of response, knowledge of correct response, and elaborate feedback[9]. Many of these studies reported that elaborate feedback resulted in higher learning gains.
  3. Worked solutions can help students recall previously acquired knowledge or help them acquire new knowledge that they can use to solve similar problems in the future[7][2].
  4. Worked solutions can help students learn concepts effectively by reducing unnecessary cognitive load from extraneous information[10][11].
  5. Students can assess their knowledge-gaps more accurately when they try to solve problems before they receive instruction, which may also help prepare them for future learning[4][5].

Data

Data was collected from the ASSISTments online learning system between September 2012 and September 2013. One of the features in the data set was students’ average frustration, which described how likely students experienced frustration while answering a problem in ASSISTments. Student frustration was predicted by an affect detector using features like the number of previous incorrect answers, time taken to solve problems, and number of hint requests[12]. An analysis of frustration instances in the ASSISTments data set showed that some problems were more challenging than others based on the small number of students who answered it correctly. The analysis also revealed an interesting behavior wherein some students continually requested for hints until it showed them the solution and the correct answer, paused for a moment, submitted the correct answer, then moved on to the next problem. The next time these students encountered a similar problem, they answered it correctly on the first attempt. Students’ usage of hints, when taken together, resembled a worked solution. This may indicate students’ preference to learn from worked solutions, and its positive effects on their learning. More details about the data used, the methodology used for analysis, and the results are presented in[13].


Related patterns

The Explain Worked Solutions design pattern closely resembles the Worked Examples design pattern[14], but provides the answer to the current problem solved by the student. The Explain Worked Solutions design pattern can be used with the Try It Yourself design pattern[6] to provide students with learning support while they answer problem-solving activities that promote better understanding of a given topic. Encoding worked solutions for problem variations may be tedious to write. Content creators may use the Mastery Learning Exercise Generator and Mastery Learning Templates design patterns to generate worked solutions automatically. It may be good idea to delay the presentation of worked solutions until students have attempted to solve the problem or tried to request basic hints as described in the Increasing Hint Specificity design pattern. The Image-Enhanced Hint design pattern[15]. can be used to clarify worked solutions. If applicable, remember to implement the One Concept Several Implementations design pattern[6] to make students aware of other solutions that may solve the problem.


Example

Many math textbooks and workbooks provide worked solutions in the back of the book so students can check their answers and see the correct solution. Several learning systems also provide students with worked solutions when they are unable to solve the problem such as ASSISTments and SQL-Tutor[16][17]. ASSISTments, in particular, uses a variety of mediums to present worked solutions such as text and image explanations, video explanations, or short video lectures[18].

Figure 1 illustrates a worked solution for a math problem about solving the sum of interior angles of triangles. The relevant concepts and the steps in the solution are explained so students can understand the process more easily. A better understanding of the problem and solution may enable them to apply this knowledge to similar problems.

Fig01 Explain Worked Solutions.png
Fig. 1. Example of a worked solution in the ASSISTments online learning system (Image courtesy of ASSISTments)


References

  1. ↑ Inventado, P.S. & Scupelli, P. (2016). Design Patterns for Math Problems and Learning Support in Online Learning Systems. In Proceedings of the 11th Viking Conference on Pattern Languages of Programs (VikingPLoP 2016). ACM: New York, NY, USA.
  2. ↑ 2.0 2.1 2.2 Sweller, J. (2004). Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture. Instructional science, 32(1-2), 9-31.
  3. ↑ D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145-157.
  4. ↑ 4.0 4.1 Loibl, K., & Rummel, N. (2014). Knowing what you don't know makes failure productive. Learning and Instruction, 34, 74-85.
  5. ↑ 5.0 5.1 Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction, 39, 72-87.
  6. ↑ 6.0 6.1 6.2 Bergin, J., Eckstein, J., Völter, M., Sipos, M., Wallingford, E., Marquardt, K., Chandler, J., Sharp, H., and Manns, M.L. (2012). Pedagogical patterns: advice for educators. Joseph Bergin Software Tools.
  7. ↑ 7.0 7.1 Hume, G., Michael, J., Rovick, A., & Evens, M. (1996). Hinting as a tactic in one-on-one tutoring. The Journal of the Learning Sciences, 5(1), 23-47.
  8. ↑ 8.0 8.1 8.2 Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of child psychology and psychiatry, 17(2), 89-100.
  9. ↑ 9.0 9.1 Mason, B. J., & Bruning, R. (2001). Providing feedback in computer-based instruction: What the research tells us. Center for Instructional Innovation, University of Nebraska–Lincoln: 14. Retrieved January 3, 2015, from http://dwb.unl.edu/Edit/MB/MasonBruning.html.
  10. ↑ Sweller, J. (2006). The worked example effect and human cognition. Learning and Instruction, 16(2), 165-169.
  11. ↑ Sweller, J., Kirschner, P. A., & Clark, R. E. (2007). Why minimally guided teaching techniques do not work: A reply to commentaries. Educational Psychologist, 42(2), 115-121.
  12. ↑ Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for Educational Data Mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487-501.
  13. ↑ Inventado, P. S., & Scupelli, P. (2015). Data-driven design pattern production: a case study on the ASSISTments online learning system. In Proceedings of the 20th European Conference on Pattern Languages of Programs (EuroPLoP 2015) (p. 14). ACM:New York.
  14. ↑ Inventado, P. S., & Scupelli, P. (2015). A Data-driven Methodology for Producing Online Learning System Design Patterns. In Proceedings of the 22nd Pattern Languages of Programs conference (PLoP 2015) (Vol. 15). ACM: New York.
  15. ↑ Inventado, P.S. and Scupelli, P. (2016). Design Patterns for Helping Students to Learn to Visualize Math Problems in Online Learning Systems. In Proceedings of the 21st European Conference on Pattern Languages of Programs (EuroPLoP 2016).
  16. ↑ Heffernan, N.T. and Heffernan, C.L. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education 24(4), 470-497.
  17. ↑ Mitrovic, A. (2012). Fifteen years of constraint-based tutors: what we have achieved and where we are going. User modeling and user-adapted interaction, 22(1-2), 39-72.
  18. ↑ Ostrow, K., & Heffernan, N. (2014). Testing the multimedia principle in the real world: a comparison of video vs. Text feedback in authentic middle school math assignments. In Proceedings of the 7th International Conference on Educational Data Mining, EDM 2014, (pp. 296-299).

External Links

Retrieved from "https://www.learningenvironmentslab.org/openpatternrepository/index.php?title=Explain_Worked_Solutions&oldid=3267"