View source for Analysis:Student affect and interaction behavior in ASSISTments

{{Infobox_analysis
|author=[[User:Pinventado|Paul Inventado]], Peter Scupelli
|dataset=[[Data:ASSISTments_2012-2013_problem-student_level|ASSISTments problem-student-level data]], [[Data:ASSISTments 2012-2013_problem_level|ASSISTments problem-level data]]
}}

Multiple studies were conducted on data collected from the ASSISTments system between September, 2012 to September, 2013. The goal of the analysis was to uncover relationships between content, student behavior, and student affect.

<div id="learningstrategies"></div>
=Student learning strategies=
Online learning systems that host student exercises and activities sometimes provide feedback in the form of hints or additional explanations. Feedback can help students figure out the answer, but can also be used for gaming the system, which hurts learning <ref>Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., and Koedinger, K. (2008). [http://www.editlib.org/p/24328/ Why students engage in “gaming the system” behavior in interactive learning environments]. Journal of Interactive Learning Research, 19(2):185-224.</ref>. Gaming is taking advantage of system feedback to get the answer without actually learning such as continuously requesting for hints or trying out all possible answers.

Such behavior was seen in [[Data:ASSISTments_2012-2013_problem-student_level | student data]] collected from ASSISTments (i.e., quickly requesting for hints to get the answer). However, further analysis showed that some students seemed to game the system to get the answer for one problem, but no longer displayed such behavior in a similar problem. It is possible that some students actually want to display all the hints not just to get the answer, but learn how to solve the problem.

=Problem content and student affect=
Relationships between content delivery design and student affect were observed. These results could be used to guide preferable design decisions and designs to avoid.

<div id="externalresources"></div>
==External resources==
An analysis of ASSISTments' [[Data:ASSISTments_2012-2013_problem_level | problem level data]] showed a correlation between the ''has_text_color'' feature, and boredom and gaming. Upon further investigation, 99% of math problems with text colors were drill and practice problems taken from textbooks. More than 70% of these problems did not contain the actual problem, but only referred students to the problems in the textbook (e.g., “Page 599 #63 Part B: Your answer should be in seconds.”). This means that when students answered these problems, they had to switch between looking up the problem in their book and using the ASSISTments interface.

According to Sweller (2004<ref name="Sweller2004">Sweller, J. (2004). [http://link.springer.com/article/10.1023/B:TRUC.0000021808.72598.4d Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture]. Instructional science, 32(1-2), 9-31.</ref>), unnecessary processing of information imposes a cognitive load that interferes with learning. High cognitive load impairs performance, which could increase the difficulty of a learning task. When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity<ref name="DMello2012">D’Mello, S., & Graesser, A. (2012). [http://www.sciencedirect.com/science/article/pii/S0959475211000806 Dynamics of affective states during complex learning]. Learning and Instruction, 22(2), 145-157.</ref>. Resulting disengagement behavior can explain the boredom and gaming behavior seen in the data.

<div id="hintusage"></div>
==Hint usage==
An analysis of ASSISTments' [[Data:ASSISTments_2012-2013_problem-student_level | problem-student level data]] showed that students rapidly requested for hints the first time they encountered the problem type. The next time the student encountered the same problem type, he/she was not involved in gaming behavior. Students could have used the hints as a worked example to find the solution<ref name="Sweller2004"/>. After learning from the hints, students no longer needed to engage in this gaming-like behavior.

<div id="language"></div>
==Language==
We use the term [https://en.wikipedia.org/wiki/Language_(disambiguation) language] to mean the style, standards, or format used when writing problem elements such as term usage, text formatting, color usage, notations, visual representations, and others. A consistent language would refer to a problem whose elements are styled and formatted the same way through out. According to ASSISTments' [[Data:ASSISTments_2012-2013_problem_level | problem level data]], frustration correlated with problems that used language inconsistently. For example, a math problem dealing with angles used the degree notation inconsistently – “Subtract the given angle from 180°. 180 - 47 = 133.” It is possible that the inconsistencies would require students to spend extra effort in trying to resolve them. When different elements are used to refer to the same thing, it will also require students to figure out mappings between these elements. This can result in a split attention effect, that increases cognitive load<ref name="Sweller2004"/>. Increasing cognitive load aside from the mental effort required to solve the problem can make the task too heavy for a student. This could lead to confusion, frustration, and disengagement from the activity<ref name="DMello2012"/>.

<div id="overpractice"></div>
=Over-practice and student affect=
Frustration is an emotion often experienced by students while learning. It is unclear whether it hinders learning or pushes students to exert more effort to learn (cf. Artino & Jones 2012<ref>Artino, A.R. and Jones, K.D. (2012). [http://www.sciencedirect.com/science/article/pii/S1096751612000073 Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning]. The Internet and Higher Education 15, 3, 170-175. </ref>; Patrick et al. 1993<ref>Patrick, B.C., Skinner, E.A. and Connell, J.P. (1993). [http://psycnet.apa.org/journals/psp/65/4/781/ What motivates children's behavior and emotion? Joint effects of perceived control and autonomy in the academic domain]. Journal of Personality and social Psychology 65, 4, 781-791.</ref>; Pekrun et al. 2002<ref>Pekrun,R., Goetz, T., Titz, W. and Perry, R.P. (2002). [http://www.tandfonline.com/doi/abs/10.1207/S15326985EP3702_4 Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research]. Educational psychologist 37, 2 (2002), Taylor & Francis, Philadelphia, PA, 91-105.</ref>). The role of frustration in learning with [[ASSISTments]] was investigated using both [[Data:ASSISTments_2012-2013_problem-student_level | problem-student-level data]] and [[Data:ASSISTments_2012-2013_problem_level | problem-level data]] that described student behavior while answering a series of questions in the system.

First, problems that likely led to student frustration were identified by ranking their average frustration confidence values. The most frustrating problems were investigated manually and revealed that they were usually difficult to understand and their hints were confusing. However, the data also showed that students were frustrated even when they answered questions correctly. 

The problem level instances were investigated further by referring to the student-level data, which described students' low level actions while answering the problem (e.g., order of hint requests and answer attempts, answer correctness, time duration between actions). Students who were frustrated even when they got the answer correctly, were observed to have spent a significant amount of time answering the exercise and consistently getting the answers correctly. This could mean that students were frustrated at the type and sequence of problems they were asked to solve rather than the problem itself.

<div id="skilllevel"></div>
=Skill level=
An analysis of [[Data:ASSISTments_2012-2013_problem-student_level | student behavior data]] showed that students reacted differently to the same problem. Students who were skilled in a certain topic rarely used hints and could quickly answer the problem. Students who were less skilled on the topic, often requested for hints, made multiple attempts to solve the problem, and spent more time before answering it correctly. Students skilled in one topic struggled with other topics, which they might not have learned as much. Without prior knowledge, students will have difficulty answering a problem<ref name="Sweller2004"/>.

Data showed that students were often bored or disengaged from the problem when they did not have enough skills to solve it (i.e., evidenced by answer correctness and number of hint requests). When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity<ref name="DMello2012"/>.

=References=
<references/>

[[Category:Analysis]]