design patterns

Paper presentation @ Viking PLoP 2017

/ scupelli

March 14, 2017 10:53 AM

Peter Scupelli will present a paper at the Viking Conference on Pattern Languages of Programs (Viking PLoP) 2017 in Schleswig-Holstein, Germany.

The conference has long roots going back to 2002 when VikingPLoP was arranged for the first time in Helsingor, Denmark. As Vikings used to travel around Europe, this year the conference heads on the shores of the Baltic Sea. VikingPLoP calls for papers on patterns and pattern languages, and papers on applying patterns.

Inventado, P.S. & Scupelli, P.: Using Contextual Learning-Environment Features to Identify Design Pattern Appropriateness

Abstract:

Pedagogical design patterns offer high-quality solutions to problems in the educational domain. Design patterns are generally written in a way that makes them applicable to multiple contexts, but how reusable are pedagogical design patterns? Over the past three years, we have tried to adapt existing design patterns and write new patterns specifically to enhance feedback in an intelligent online learning system for Math called ASSISTments. However, this has proved to be difficult because there appear to be features of learning environments that call for patterns that are either too general or too specific. For example, design patterns whose context involves interpreting learners’ misconceptions may be easy for teachers in traditional classroom settings, but difficult for intelligent learning systems because algorithms that predict misconceptions are currently imperfect. In this paper, we identify contextual learning-environment features and investigate how they might affect the appropriateness of design patterns to a given learning environment.

Last updated: 5:46 pm

Paper presentation @ EuroPLoP 2017

/ scupelli

March 14, 2017 11:09 AM

Paul Inventado will present a paper at the 22nd European Conference on Pattern Languages of Programs (EuroPLoP) 2017 in Kloster Irsee, Bavaria, Germany.

EuroPLoP is the premier European conference on patterns and pattern languages. Design patterns are a unique and effective way to capture and share expertise, tacit knowledge and research findings.

Inventado, P.S., Scupelli, P., Heffernan, C. & Heffernan, N.: Feedback Design Patterns for Math Online Learning Systems

Abstract:

Increasingly, computer-based learning systems are used by educators to facilitate learning. Several learning systems have been developed for the Math domain, which have resulted in significant improvements in student learning. Feedback provision is one of the key features in Math learning systems that contribute to its success. We have recently been uncovering feedback design patterns as part of a larger pattern language for Math problems and learning support in online learning Systems. In this paper we present three feedback design patterns developed from the application of the data-driven design pattern (3D2P) methodology on a large educational data set collected from actual student data in a Math online learning system. These design patterns can help teachers, learning designers, and other stakeholders construct effective feedback for interactive learning activities that facilitate student learning.

Last updated: 5:45 pm

Paper presentations @ EDM2016

/ scupelli

June 22, 2016 11:34 AM

Peter Scupelli and Paul Inventado presented papers at The 9th International Conference on Educational Data Mining (EDM2016) in Raleigh, North Carolina.

1. Inventado, P.S., Scupelli, P., Van Inwegen, E., Ostrow, K. Heffernan, N. , Baker, R.,  Slater, S., and Ocumpaugh, J. “Hint Availability Slows Completion Times in Summer Work”

Abstract:

On-demand help in intelligent learning environments is typically linked to better learning, but may lead to longer completion times. This present work provides an analysis of how students interacted with a summer learning assignment when on-demand help was available, compared to when it was not. When hints were available from the start, students were more likely to delay work, compared to students for whom step-wise hints were only available after the third problem. When hints were always available, participants took significantly more time to complete a mastery learning assignment,. We interpret this difference in time to complete the assignment as an opportunity to re-engage in productive math learning.

2. Slater, S., Ocumpaugh, J., Baker, R., Scupelli, P., Inventado, P.S., and Heffernan, N. “Semantic Features of Math Problems: Relationships to Student Learning and Engagement”

Abstract:

The creation of crowd-sourced content in learning systems is a powerful method for adapting learning systems to the needs of a range of teachers in a range of domains, but the quality of this content can vary. This study explores linguistic differences in teacher-created problem content in ASSISTments using a combination of discovery with models and correlation mining. Specifically, we find correlations between semantic features of mathematics problems and indicators of learning and engagement, suggesting promising areas for future work on problem design. We also discuss limitations of semantic tagging tools within mathematics domains and ways of addressing these limitations.

Last updated: 11:34 am

Paper presentation @ EuroPLoP 2016

/ scupelli

June 22, 2016 1:00 AM

Peter Scupelli and Paul Inventado presented their paper, “Media-type Selection Design Patterns for Problem-solving Content and Support in Online Learning Systems” at the 21st European Conference on Pattern Languages of Programs (EuroPLoP) 2016 in Bavaria, Germany.

EuroPLoP is the premier European conference on patterns and pattern languages. Design patterns are a unique and effective way to capture and share expertise, tacit knowledge and research findings.

Abstract:

Online learning systems have been gaining popularity, but are not without their challenges. For example, enrollment in MOOCs has slowed down, which is attributed to the lack of sustainability. Research has also shown that relying on delivered content alone results in lower learning gains. However, introducing learning activities increases learning gains as much as six times. These results emphasize the importance of designing high quality learning activities for online learning systems. Although there are many design patterns that may be applied in designing learning activities, they usually operate at a higher level. There is a need for design patterns that address problems in implementing these learning activities. This paper presents four design patterns that focus on helping students learn to represent math problems properly in the context of math online learning systems. These patterns can guide online learning system stakeholders (e.g., system developers, content creators, teachers) in creating high quality learning activities in online learning systems.

Last updated: 1:00 am

Paper presentation @ Viking PLoP 2016

/ scupelli

April 11, 2016 8:10 AM

Peter Scupelli and Paul Inventado presented  “Design Patterns for Math Problems and Learning Support in Online Learning Systems” at the Viking Conference on Pattern Languages of Programs (Viking PLoP) 2016 in Leerdam, Netherlands.

Abstract:

Increasingly, many institutions and students benefit from online learning systems each year. For example, in 2016 Massive Open Online Courses (MOOCS) reported as many as 16 million enrolled students and online tutoring systems reported over half a million enrolled students. In the literature, many design patterns capture online learning system designs for class management, discussion facilitation, lecture delivery, and feedback. In this paper, we describe design patterns that describe finer-grained activities within online learning systems such as the design of problem-solving activities and their associated learning support. The three patterns presented in this paper describe designs for constructing math-problem content and corresponding learning support for students who answer these problems – Mastery Learning Templates, Explain Worked Solutions, and Scaffold Problems with Guide Questions. We found these patterns using the data-driven design pattern production (3D2P) methodology on data collected from the ASSISTments online learning system. The design patterns we describe were mined from data on student interactions with an online learning system and linked patterns to existing learning science literature.

Last updated: 8:10 am

Paper presentation @ PLoP 2015 writer’s workshop

/ scupelli

October 27, 2015 3:36 PM

Paul Inventado and Peter Scupelli presented “A Data-driven Methodology for Producing Online Learning System Design Patterns” at the 22nd Conference on Pattern Languages of Programs (PLoP) 2015 in Pittsburgh, Pennsylvania.

Abstract:

Online learning systems are complex systems that are frequently updated with new content, upgraded to support new features and extended to support new technologies. Maintaining the quality of the system as it changes is a challenge that needs to be addressed. Design patterns offer a solution to this challenge by providing guides to stakeholders responsible for making design changes (e.g., system developers, HCI designers, teachers, students) that will help them ensure system quality despite changes. Although design patterns for online learning systems exist, they often focus on one aspect of the system (e.g., pedagogy, learning). The data-driven design pattern production (3D2P) methodology utilizes data for producing design patterns in collaboration with stakeholders, addresses stakeholders’ concerns, and ensures the system’s quality as a whole. The paper presents five patterns produced by applying the methodology on the ASSISTments online learning system namely: all content in one place, just enough practice, personalized problems, worked examples, and consistent language. We made two changes to the pattern format: added in-text references in the forces section, and added an evidence section. The references allow the reader to learn more about the force in question. The evidence section highlights key findings uncovered from the 3D2P methodology. Four sources of evidence were considered in the pattern format: (a) literature – existing research on the problem or solution, (b) discussion – expert opinions about the problem or solution, (c) data – measures of the problem’s recurrence, and the solution’s effectiveness based on collected data; and (d) evaluation – assessment of the pattern’s performance when it was applied on an existing system. The changes to the format highlight linkages between pattern elements, theory, and empirical evidence. We believe that links further justify the design pattern, and make it easier for multiple stakeholders to understand them.

Last updated: 3:36 pm

Focus group @ PLoP 2015

/ scupelli

October 27, 2015 3:32 PM

Paul Inventado and Peter Scupelli led a focus group discussion on “Developing an open, collaborative design pattern repository” at the 22nd Conference on Pattern Languages of Programs (PLoP) 2015 in Pittsburgh, Pennsylvania.

Abstract:

Ensuring the production of high quality patterns is challenging for four reasons: first, it is expensive (e.g., face-to-face meetings in international conferences); second, it lacks incentive for evaluating, critiquing, improving, and evolving existing pattern languages; third, author attribution is an issue; and fourth, stakeholders, especially designers and end-users who are the primary beneficiaries of patterns, are often not part of the process (Dearden & Finlay, 2006). There have been calls for a widespread collaboration between stakeholders in the production of design patterns (Bayle et al., 1998; Dearden & Finlay, 2006). In this workshop, we present a framework for collaboratively producing design patterns. We use the case study of producing online learning system design patterns to explain the open, collaborative pattern repository. We describe how diverse stakeholders (e.g., educators, learning analytics experts, interaction designers, online learning system experts and developers, pattern writers) could contribute to the design pattern production process. We use an open pattern repository Wiki prototype to illustrate the concept. The repository would manage design patterns and the processes needed for producing and maintaining them such as indexing, versioning, documentation, and communication.

Last updated: 3:32 pm

Poster presentation @ Learning with MOOCs II – 2015

/ scupelli

October 16, 2015 8:29 PM

Paul Inventado and Peter Scupelli presented “Addressing MOOCs’ Sustainability Issues Using Data-driven Design Pattern Production” at Learning with MOOCs II – 2015 in Teachers’ College Columbia University, New York.

Abstract:

Despite the popularity of MOOCs, some higher education institutions have started moving away from it. Academic leaders seem to be less convinced of MOOCs’ sustainability. Four challenges need to be addressed to improve MOOC sustainability: (a) representing and communicating MOOCs best practices, (b) evaluating MOOC quality, (c) testing, validating, and refining MOOC designs, and (d) fostering research and collaboration around MOOCs. The data-driven design pattern production (3D2P) framework discussed here, can be used to address these four issues and can lead to the development of more sustainable MOOCs.

Last updated: 8:29 pm

Virtual presentation @ Learning Analytics Workshop

/ scupelli

July 31, 2015 6:04 PM

Paul Inventado virtually presented “Promoting Online Learning System Design Quality: Utilizing Design Patterns Produced by Data-driven Approaches” at the Learning Analytics workshop in Prague, Czech Republic.

Abstract:

Many students benefit from online learning systems each year. However, it is not easy to ensure the design quality of these systems due to their complexity. In this paper, the data-driven design pattern production (3D2) methodology is presented as a solution. Specifically, it uses learning analytics and educational data mining to help uncover relationships between student learning outcomes and system designs. Designs that lead to better learning can be formalized into design patterns, which stakeholders can use to guide them in upgrading the online learning system’s components, and adding new content. The approach is further extended into an open, collaborative framework, which allows stakeholders to collaborate in the production of design patterns. A collaborative effort can speed up the pattern production process, improve the quality of the design patterns produced, share benefits among all members, and ultimately, elevate the standards of online learning system development.

Last updated: 6:04 pm

Paper presentation @ EuroPLoP 2015 writer’s workshop

/ scupelli

July 11, 2015 3:54 PM

Peter Scupelli and Paul Inventado presented “Data-Driven Design Pattern Production: A Case Study on the ASSISTments Online Learning System” in a writing workshop 20th European Conference on Pattern Languages of Programs (EuroPLoP) 2015 in Bavaria, Germany.

Abstract:

Online learning systems popularity increased rapidly in recent decades in multiple domains such as cognitive tutors, online courses, and massive open online courses (MOOCS). The design quality of online learning systems is difficult to maintain. Multiple stakeholders are involved (e.g., software developers, interaction designers, learning scientists, teachers), the system is complex, there are rapid changes in software, platforms (e.g., mobile, tablet, desktop) and learning subject content, and so forth. Many existing online learning systems collect a significant amount of data that describe learning outcomes and student behaviors, which are indirect measures of system quality. Data analysis on online learning systems data can uncover linkages between particular design choices made and student learning outcomes. In this paper, we describe the Data-Driven Design Patterns Production (3D2P) methodology to prospect, mine, write and evaluate design patterns for online learning systems. Pattern prospecting helps designers decide what type of possible meaningful outcomes and features to scan for in the data and helps to focus on specific data subsets to limit the search space for pattern mining. Design patterns identified with 3D2P methodology can guide the addition of new content and the modification of system designs to maintain the online learning system’s quality. We present a case study of the ASSISTments math online learning system to illustrate the 3D2P methodology and discuss its benefits and limitations.

Last updated: 3:54 pm