Making Math Tutors More Engaging and Effective through Interaction Design Patterns and Educational Data Mining

Ryan Baker, Neil Heffernan and Peter Scupelli

The growth in the American economy is linked to industries based upon science, technology, engineering, and mathematics (STEM). Unfortunately, American students lack the degree of proficiency in mathematics needed to pursue careers in these areas. Increasingly, online software for mathematics problem-solving — whether used in class or as homework — is used by hundreds of thousands of students each year. However, the quality of student learning experiences can differ considerably between different content, even when the content is developed largely by the same team. In recent years, researchers have attempted to formalize lessons learned from past successful design projects into design principles. These efforts have been instructive, but there are several challenges to instantiating design principles in new contexts and many cases where principles may conflict. In this research, we use Design Patterns, an alternate approach which has been successful in other domains which can be used at scale as templates for realizing effective design (i.e., architecture, programming, security). We are using a data mining approach to extract the design patterns in addition to traditional design pattern development approaches. The design patterns are used to fix math problems associated with poor learning outcomes. The revised math problems are then tested with online experiments to ensure that the design pattern lead to improved learning outcomes.

Publications

Inventado, P. S., Scupelli, P., Ostrow, K., Heffernan, N., Ocumpaugh, J., Almeda, V., & Slater, S. (2018). Contextual factors affecting hint utility. International Journal of STEM Education, 5(1), 13.

Inventado, P. S., Inventado, S. G. F., Matsuda, N., Li, Y., Scupelli, P., Ostrow, K., Heffernan, N., Tu, S., Mason, C., Logue, M., & McGuire, P. (2018). Using Design Patterns for Math Preservice Teacher Education. In Proceedings of the 23rd European Conference on Pattern Languages of Programs Irsee, Germany — July 04 – 08, 2018. ACM. New York, NY, USA, 31.

Inventado, P. S. & Scupelli, P.(2017). An Online Learning Collaboratory to Address Multidisciplinary Learning Challenges at Scale. Interaction Design and Architectures, (33), 11-32.

Inventado, P.S. & Scupelli, P., Heffernan, C., & Heffernan, N. (2017) Feedback Design Patterns for Math Online Learning Systems.  2017 Conference Proceedings of the European Pattern Languages of Programs. (Euro-PLoP 2017) ACM, New York.  [cited 1 time]

Inventado, P. S., & Scupelli, P., (2017). Adapting design patterns across learning environments. In Proceedings of the VikingPLoP 2017 Conference on Pattern Languages of Program (p. 10). ACM.

Inventado, P.S. & Scupelli, P. (2016). Patterns for Learning Support Design in Math Online Learning Systems. In Proceedings of the 24th Conference on Pattern Languages of Programs (PLoP 2016). ACM, New York. [cited 2 times]

Inventado, P.S., Scupelli, P., Köppe, C. , & Van Heesch, U. (2016). Towards Extending Online Pattern Repositories: Supporting the Design Pattern Lifecycle. In Proceedings of the 22nd Conference on Pattern Languages of Programs (PLoP 2016). ACM, New York.

Inventado, P.S. & Scupelli, P., (2016) Design patterns to help students represent math problems within online learning systems.  2016 Conference Proceedings of the European Pattern Languages of Programs. (Euro-PLoP 2016) ACM, New York, NY, USA.

Inventado, P.S. & Scupelli, P. (2016). Design Patterns for Math Problems and Feedback in Online Learning Systems. In Proceedings of the 11th Viking Conference on Pattern Languages of Programs (VikingPLoP 2016). ACM, New York, NY, USA. [cited 3 times]

Slater, S., Ocumpaugh, J., Baker, R., Scupelli, P., Inventado, P.S., and Heffernan, N.  (2016) Semantic Features of Math Problems: Relationships to Student Learning and Engagement. In The 9th International Conference on Educational Data Mining EDM 2016 Raleigh, North Carolina, USA, June 29, 2016 – July 2, 2016.

Inventado, P. S., Scupelli, P., Van Inwegen, E., Ostrow, K., Heffernan III, N., Ocumpaugh, J., Baker, R., Slater, S., & Almeda, M. (2016). Hint Availability Slows Completion Times in Summer Work. In The 9th International Conference on Educational Data Mining June 29–July 2, 2016, Raleigh, NC, USA. (pp. 388-393).

Inventado, P. S. and Scupelli, P. (2015). Towards an open, collaborative repository for online learning system design patterns. eLearning Papers, 42(Design Patterns for Open Online Teaching):14-27. 

Inventado, P.S. and Scupelli, P. (2015). Data-Driven Design Pattern Production: A Case Study on the ASSISTments Online Learning System. In Proceedings of the 20th European Conference on Pattern Languages of Programs (EuroPLoP 2015). 

Inventado, P. S. and Scupelli, P. (2015). Promoting Online Learning System Design Quality: Utilizing Design Patterns Produced by Data-driven Approaches. Learning Analytics Summer Camp, Prague, Czech Republic. 

Inventado, P.S. and Scupelli, P. (2015). A Data-driven Methodology for Producing Online Learning System Design Patterns. In Proceedings of the 22nd Conference on Pattern Languages of Programs (PLoP 2015).

Scupelli, P. and Inventado, P.S., (2014) Data-driven Design Pattern Development (3DPD) Workshop in Pattern Languages of Programs Conference 2014 September 14-17, 2014 (PLOP 2014) Allerton Park in Monticello, IL.  Read more about the project

Online learning system pattern repository

Peter Scupelli and Paul Inventado

Many students benefit from online learning systems each year. However, it is not easy to ensure the design quality of these systems due to their complexity. In this research, we explore the use of the data-driven design pattern production (3D2) methodology to address this problem. Specifically, the 3D2P methodology uses learning analytics and educational data mining to help uncover relationships between student learning outcomes and system designs. Designs that lead to better learning can be formalized into design patterns, which stakeholders can use to guide them in upgrading the online learning system’s components, and adding new content. The approach is further extended into an open, collaborative framework, which allows stakeholders to collaborate in the production of design patterns. A collaborative effort can speed up the pattern production process, improve the quality of the design patterns produced, share benefits among all members, and ultimately, elevate the standards of online learning system development.

Visit the Open learning system pattern repository website to learn more.

Publications

Inventado, P. S. and Scupelli, P. (2015). Towards an open, collaborative repository for online learning system design patterns. eLearning Papers, 42(Design Patterns for Open Online Teaching):14-27. 

Classroom Environment, Allocation of Attention, and Learning Outcomes in K-4 Students

Anna Fisher, Ryan Baker, Howard Seltman, Sharon Carver and Peter Scupelli

Distractibility decreases markedly with student age. First, classrooms for younger learners usually contain more potential sources of visual distraction (e.g., art work, posters, alphabet charts, etc.) than older student’s learning environments. Do educational materials that are not directly relevant to the ongoing instruction present a distraction for young learners? And if so, does off-task behavior related to the classroom visual environment affect learning outcomes? Second, the classroom environment is much more malleable than other factors related to learning such as the students’ socio-economic status, aptitude, perseverance, and so forth.This research is focused on the following research question: How might the visual environment in K-4th grade classrooms contribute to off-task behavior in K-4 students? The LEL lab is focusing on documenting the contents of the classroom visual environment and developing design guidelines to decrease visual clutter in the classroom environment.

Publications

Almeda, M. V., Scupelli, P., Baker, R., Weber, M. , Fisher, A. V. (2014). Clustering of design decisions in classroom visual displays. LAK 2014, pp. 44-48.
Fisher, A.V., Godwin, K.E., & Seltman, H. (in press) Visual environment, attention allocation, and learning: When too much of a good thing may be bad. Psychological Science.
Godwin, K.E., Almeda, M. V., Petroccia, M., Baker, R.S., & Fisher, A.V. (2013). Classroom activities and off-task behavior in elementary school children. In M. Knauff, M. Pauen, N. Sebanz, and I. Wachsmuth (Eds.), Proceedings of the 35th Annual Meeting of the Cognitive Science Society, 2428-2433.Design Studio Learning Environments

Peter Scupelli and Bruce Hanington

Studio-based design education is changing to include multidisciplinary design teams, geographically distributed teams, information technology, and new work styles. The design studio in universities is a physical place, a cultural place, and a social place. Activities are characterized by five factors: co-location, learning-by-doing, continuous access, integrative learning, and mimicking practice (Lawson & Dorst, 2009). A studio-based education ecological framework includes observable components, or “tools”, and pedagogical approaches used to construct design knowledge (Brandt, Cennamo, Douglas, Vernon, McGrath, & Reimer, 2011). In this ongoing research we are investigating the role of the design studio in design education.

Publications

Scupelli, P., & Hanington, B. (2014). An Evidence-Based Design approach for function, usability, emotion, and pleasure in studio redesign. Proceedings of the Design Research Society.

Design Studio Pedagogy

Peter Scupelli

Increasingly, designers are engaged in societal areas of human concern. For example, environmental degradation (e.g., Ortbal, Lange, Carroll, & AIGA, 1996), toxic chemicals (e.g., McDonough & Braungart, 2002), climate change (e.g., IDEO, 2009), voting rights (e.g., Lausen 2007), and so forth. The design methods movement explained the need for new design methods to deal with the increased complexity of societal concern (e.g., Jones, 1992). Societal areas of human concern are described as wicked problems that cannot easily be reduced to simple design problem (Rittel & Weber, 1973). Many design disciplines are engaged with societal challenges, examples include: architecture for humanity (2006), industrial design (e.g., Papanek, 1970), communication design for good (e.g., Berman, 2009), organizational change (Brown, 2009), instructions for continued life on planet earth (Fuller, 1969), and so forth. Design action within such areas of societal concern is complex and broad in scope and often requires multiple design strategies to find actionable design opportunities.

In this ongoing research we are investigating design studio pedagogy to account for the new challenges designers face.

Publications

Scupelli, P. & Rohrbach, S. (2013). Speak Lab: Using social change storytelling to teach design agility. Proceedings of the AIGA BLUNT conference.