Multi-Level Assignment/alx

Multi-Level Assignment
Contributors	Christian Köppe, Leo Pruijt
Last modification	May 16, 2017
Source	Köppe & Pruijt (2014)
Pattern formats	OPR Alexandrian
Usability
Learning domain
Stakeholders

Context. You want to design an assignment that supports meaningful learning of specific software engineering concepts. You already introduced some basic factual, conceptual, and procedural knowledge on the concepts.

Problem. Student learning is suboptimal when assignments can be completed by merely applying techniques and concepts without requiring a deeper conceptual understanding of them. In that case the students might learn less than they could have.

The deep understanding of a concept requires knowledge and skills at different levels, but different levels often require different educational methods. The procedural level—students can apply the concepts—is often implemented by giving the students work assignments, e.g. programming a certain tool, gathering and documenting requirements, or defining the architecture of a complex system.

Forces. Students should apply the techniques they have to learn, but often do that through copying and pasting from existing solutions and adjusting these so that they fulfill the requirements. Even though the resulting solutions might be sufficient, this often does not require the desired deeper understanding of the concepts we want the students to learn.

Trial and error, an approach often used by students, can lead to sufficient results too, but does not necessarily require conscious thinking about and awareness of the applied concepts. If students also experience this trial and error approach as sufficient, they are not supported in gaining a deeper understanding of the concepts.

Assignments often require one concrete application of concepts, which makes it hard to generalize these concepts for the students.

Extending assignments with exercises that also explicitly cover cognitive processes regarding the conceptual knowledge leads to parallel or sequential activities, which makes it harder for the students to see the connection between the procedural and conceptual parts of the concepts.

Solution. Therefore: Make the concepts the students have to study (part of) the assignment domain itself and not only part of the techniques they have to apply in order to realize the assignment.

This way the students have to apply the concepts when realizing the assignment and also have to understand the concepts on a higher level as part of the application domain. This could be even more improved by having the students also analyze (parts of) this domain as part of the requirements gathering and make them using some realistic data for testing and demonstrating purposes.

The following educational objectives (including their corresponding cells in Table I below) are tackled by such an assignment (”the students are able to”):

—Remember previously taught essential knowledge of all categories (A1 to C1).

—(optional) Analyze the domain of the assignment and use this as basis for the domain model (B4; also A4 if known examples are used as basis for the analysis).

—Interpret and classify the concepts in the domain model (B2) and use specific instances as sample data for the application (A2).

—Evaluate the domain model before it’s used as basis for the design of the application (B5).

—Apply the procedural knowledge for the realization of the concepts in the assignment (C3), which requires an understanding of how to apply them (C2).

—Plan and optionally think of alternative ways of implementing the product and applying the concepts to be learned (C6).

—Implement the assignment which certainly requires the creation of some larger product that contains the concepts (B6).

The broad coverage of educational objectives shows that indeed a higher learning target can be reached by using Multi-Level Assignment (Multi-Level Assignment). According to Raths this can also be interpreted as evidence for improved instruction. Please note: Metacognitive knowledge and its related cognitive processes are not covered with multi-level assignments.

Table I. : Mapping of the educational objectives of MULTI-LEVEL ASSIGNMENT to the revised Bloom’s taxonomy

Implementation. One important part is to identify upfront the concepts you want the students to learn on different levels. Setup the assignment so that (a) the concepts (or the most important parts of them) are (part of) the domain of the assignment and have to be described as domain model and (b) that the students also have to apply the concepts on a procedural level. Making the concepts part of the domain implies that the functional requirements of the assignment are in line with how the concepts would be applied in “real” projects. In most cases this could be an administration tool for specific instances of the concepts, like a software requirements administration (that is itself defined by requirements) or an architecture administration/visualization tool (that has itself an architecture). Another option is a generation tool, that generates concrete instances of previously defined implementation independent instances of concepts, e.g. a business rule generator.

As described in the solution part, the learning target can be broadened by adding the cognitive process of analyzing to the assignment. In that case require the students to analyze (parts of) the assignment domain—including the concepts to be learned—mostly by themselves as part of the functional requirements. You may provide an initial version as starting point, mainly for triggering the thought process and making the first steps easier.

The assessment of such an assignment is similar to that of assignments with other domains. Of main importance is that the domain model has to be correct and consistent, which means for a Multi-Level Assignment (Multi-Level Assignment) that the conceptual knowledge is much more emphasized than in typical software engineering assignments.

Consequences. Even though the students still can copy&paste and adjust an existing solution when realizing the assignment, they are also forced to have to work on and with the concepts on a different—the conceptual—level.

Applying trial and error might still work for the technical realization of the assignment, but it does not help with the conceptual component of the assignment: the definition and realization of the domain does not allow trial and error.

As the students are exposed to the concepts they are applying on a conceptual level too, it probably will be easier for them to recognize the generic aspects of them.

In some cases, like interdisciplinary assignments or when involving industry partners, pre-determining the assignment domain does not work as it does not fit the needs of the involved stakeholders. In such cases the pattern is not applicable.

There is also the chance that the students would favour another assignment domain which is closer to their interests, like e.g. a specific game or a gym administration.

Example: Business Rule Generator. In the second year of the bachelor program computer science at the HU University of Applied Sciences, we teach the students to recognize, specify, classify and implement business rules as part of a course on Software Requirements. The Business Rules Group defines a business rule (from the information system perspective) as “a statement that defines and constraints some aspect of a business. It is intended to assert business structure, or to control or influence the behavior of the business.” In our program, we introduce the basics of business rules, but focus thereafter on data rules and how they are classified according to the BCDM framework.

The project assignment was developed in cooperation with a software development company specialized in Oracle Technology. It stimulates the students to think and work with business rules at both procedural and conceptual level. The students are instructed to design and build a business rule generator. Their resulting generator should allow a software developer to enter the functional details of a business rule via the GUI component of their application, which means that business rules are the assignment domain. On demand the rule should be transformed automatically, by the generator component, to a database constraint or database trigger in the SQL-variant of a pre-selected database system.

Functional requirements are provided to the students, including the details of nine types of rules and examples of the rule types with related code examples to implement the rule as database constraint or trigger. Before the students start building the generator, they have to design a conceptual model that answers the requirements. Furthermore, they need to specify and implement an accompanying set of business rules for their generator, constraining the data to be entered into their system—the procedural part of the pattern solution. By doing so, the students improve their analytical skills and develop a meta model of the business rule domain, which adds to their knowledge and is the result of the pattern application.

Figure 1 shows a screenshot of one of the student systems. All metadata relevant for specific business rule types have to be defined through the tool including business rules constraining them. These rules then have to be checked when entering specific business rules for later generation.

Fig. 1: Student system example: Business rule definition (the selection of a specific business rule type implies specific business rules to be checked for this type, e.g. a limited set of available operators and allowed selections of attributes)

Example: Architecture Compliance Checking Tool. Software architecture is an important topic within our bachelor program computer science. At the end of the second year, preceded by several analysis and design modules, an introduction in software architecture is provided, and in the third year we offer a module Advanced Software Architecture and a related, large project assignment. The basics of software architecture, modular design, layers, components and architectural patterns are discussed at first, with the focus on the design of semantically rich modular architectures. Next, to develop the practical skills, assignments are made in which the students classify different types of responsibility, design layers and components at the logical and physical level, and design application scenarios in accordance to the architectural models and/or in accordance to architectural patterns.

The third-year project assignment invites the students to think and work with software architecture at different levels. The assignment focuses on tools to support checks on architecture compliance; “a measure to which degree the implemented architecture in the source code conforms to the planned software architecture”. Previous to the assignment, the students have to identify requirements to this type of tools and use and test different tools. Next, the students work on the design and development of a new open source software architecture compliance checking tool. A domain model is used to identify and relate the most important concepts of architecture compliance including different software architecture elements, their relations and rules applying on these relations. Furthermore, a component model is used to identify and relate the functional parts of the tools, to allocate the domain concepts to these parts, and to divide the work over (up to six) student teams. Each team is responsible for one component and the teams have to negotiate the requirements with regard to the provided services, as well as the implementation details. That way the students had to apply the same software architecture concepts for building the tool that also are part of the domain of the tool, which is the core of the pattern.

At the end of the semester, one-day architecture compliance checks were organized at the sites of software development organizations. On such a day, a professional application was analyzed with the help of the tool, and checks were performed on the conformance to the software architecture. At the end the students had to present suggestions for improvement regarding the actual realization of the software architecture in the analyzed applications. These suggestions were well perceived by the present software architects of the companies and demonstrated a high conceptual understanding of the architectural concepts and their realizations in applications by the students. We interpret this as indication that the pattern application indeed was successful.

Example: Design Pattern Selector. We have applied the pattern for the design of an assignment for the course “Patterns & Frameworks”, part of the second year of an undergraduate computer science program with a focus on software engineering. The main requirement for the assignment was to implement a pattern selector and a pattern editor in Java. The students therefore had to apply design patterns for the design of the tool while the (same) design patterns also formed the domain of the tool. This reflects the core of the pattern solution.

In that example we went through all educational objectives as described in the pattern solution and how they were included in the assignment. This helped us to consciously check the coverage of objectives and demonstrates the practical support the pattern solution offers for assignment design. Based on this check we decided to add some more focus on specific educational objectives, namely understanding and analyzing conceptual knowledge. We did so by requiring that the tool should be usable for all domains of design patterns (instead of only the GoF-patterns as presented in) through the usage of generalized pattern description sections (context, problem, forces, etc. instead of intent, motivation, etc.). The assignment furthermore included that the students had to fill their repository with at least ten GoF-patterns. The students hereby had to interpret and classify the pattern parts by themselves and to organize these parts in the given structure.

A set of extra functional and non-functional requirements was given that was related to typical problems addressed by specific GoF design patterns. An example is that one could add patterns onthe-fly while leaving the selection user interface open, and that the selection lists automatically are updated after adding a new pattern or a new context (solvable with the OBSERVER pattern). Another example is that the repository of patterns that one builds up should be exportable and importable in different formats, and these formats should be easy to add (solvable with the Factory Method (Factory Method) and Strategy (Strategy) patterns).

Finally, the requirements included that the tool explicitly has to support the method of how patterns should be selected prior to applying their solution, which is an application of the educational pattern Context, Problem and Consequences First (Context, Problem and Consequences First). One should first check if the context the pattern describes and the design context are sufficiently matching. Then the students have to analyze what the actual design problem is that they want to solve. Next, the problem that needs to be solved should be matched with the problems that can occur in the selected context (a list generated from the pattern descriptions). Finally, before presenting the pattern solution, one should first carefully look at the consequences the application of the pattern solution has to trade off benefits against liabilities.