Cog-Learn: An e-Learning Pattern Language for Web-based Learning Design (01/11/2009)

http://www.elearnmag.org/subpage.cfm?section=case_studies...

[See page 4 for comments]

By Junia Coutinho Anacleto, Americo Talarico Neto, and Vania Paula de Almeida Neris,
Federal University of Sao Carlos, Brazil

August 4, 2009

Abstract
Designing online learning material is a difficult task for novice teachers who lack experience in their design. Patterns have emerged as means to capture design knowledge in context and offer solutions to designers.

Cog-Learn is a pattern language aimed at supporting the design of learning material for e-learning systems. Here, we describe Co-Learn and discuss the patterns' identification and formalization processes through two case studies in which a set of cognitive strategies was applied with the goal of better organizing the content seen by the student. The purpose is to facilitate the student's interaction with the material's interface and, consequently, improve the learning process.

Introduction
Designing Web-based content for e-learning is a difficult task for novice teachers who lack experience in interaction and learning design for the electronic environment. The results are poorly designed courses and learning contents—for instance, text documents with too much information, which hinder the students' learning [10].

This research, supported by TIDIA-Ae project from FAPESP (process 03/08276-3), aims at designing learning material for Web-based e-learning and considers the different characteristics and knowledge of the multidisciplinary group that interact in such a project. We synthesize the cognitive science's proposals, expressed here as a set of cognitive strategies adopted by Liebman [11], some of them from Ausubel [2], and some concepts used during interaction projects on Web systems such as universal design, participative design, and accessibility. We have documented those practices in patterns to support the design of the learning material.

Considering such patterns, we propose to generate a common vocabulary among the participants of the multidisciplinary group that are responsible for designing the learning contents for e-learning (such as teachers, authors, educators, interface designers, software engineers, and Web designers), separating common qualities of existent designs, identifying successful solutions, and presenting the relevance of such solutions to help teachers better organize the content and thus benefit the students who are going to use it.

Here "teacher" is the professional responsible for designing the e-learning material, while "student" means the user that will interact with the developed Web interface published as learning content.

This article is divided as follows. The first section briefly presents the cognitive strategies theory and the group that we used in the case studies.

Second, we present the patterns and pattern language concepts.

Third, we show the methodology used to conduct the case studies, including the framework that guided the usability evaluations.

Fourth, we show the results from the case studies.

Fifth, we present the e-learning pattern language identified, its details, and some potentiality and restrictions.

Finally, we introduce a pattern-based tool to support the design of learning material, and end with some conclusions.

Cognitive Strategies to Support Teaching
Cognitive strategies are internally organized capacities that students use to guide their attention and manage their learning process. Gagné [8] relates these strategies to the "learn to learn" and "learn to think" concepts. Based on these ideas, researchers such as Beckman [3] and Ausubel [2] have thought about how to help students work with this information, which means using these strategies to facilitate understanding and information retention.

We adopted the following cognitive strategies used by Liebman [11]: organizing, framing, concept maps, metaphors and analogies, rehearsals, and advance organizers.

Another connection between our work and Liebman's [11] is the acknowledgement that teachers can use the cognitive strategies to facilitate the student's learning process. Teachers select and use the strategies when designing the learning material. The goal is to better organize the content through the interface and facilitate the student's learning process.

Patterns to Support the Design of Learning Material for E-Learning
Patterns were used first in the architecture domain [1] to represent success solutions for recurring problems found on this context. A pattern can be understood as an approach to capture and present design knowledge in problem resolution—acting as a knowledge spread tool between the specialist and inexperienced designers, and as a communication tool for the team members.

A pattern usually exists within a pattern language, and it is related to other patterns that offer solutions for other design issues in the same domain, aiming at involving the final user in all the stages of the software development process [5].

A pattern language which supports the learning material design for e-learning must have patterns to guide teachers in how to plan a lecture structure, as well as how to organize its content. The patterns must also help them plan the sequence of students' actions and provide support during the course, besides considering interaction design questions such as navigation and layout [16].

In view of that, we proposed a pattern language aimed at supporting teachers who design learning content and publish it in a learning management system. In this context, the teacher performs the user role when using the pattern language as a tool to establish more efficient communication and participate more actively in multidisciplinary learning design. She or he takes on the designer role when designing, evaluating, and making the learning material available, using the pattern language as a tool, which also conveys knowledge to less experienced teachers.

The patterns were obtained through two case studies whose main goals were 1) to verify if a set of six cognitive strategies [11] increases the usability of the learning material, and 2) if they can be considered solutions for recurring problems in this context and, this way, be written in a design pattern form.

The patterns' form and writing style are based on research by Meszaros and Doble [12], considering the elements name, forces, context, problem, solution, reasoning, examples, and related patterns.

Case Studies
The research method adopted by us is the case study [6] with qualitative analysis based on observation and questionnaires. We choose this method because if the study is not perfect, it is possible to obtain good results using qualitative methods that are based on users and their behavior [15].

The hypotheses used as a starting point to guide the case studies are:

Hypothesis 1. The use of six selected cognitive strategies improves the structure and organization of the content that will be published electronically for the student, and consequently, increases its usability.

Hypothesis 2. The cognitive strategies can be solutions to recurring problems in the Web-based learning design context and, thus, one can then write in a pattern form.

The method used to prove Hypothesis 1 is the accomplishment of two case studies, performing usability evaluations of the interface of two types of learning materials, one using the cognitive strategies and another not using them, in an effort to verify if there are usability issues in the materials designed without the cognitive strategies and if these issues were minimized in the materials designed with cognitive strategies.

The usability evaluations were planned according to the D.E.C.I.D.E. framework [17], which aims to support the planning and accomplishment of a usability evaluation. It was chosen using both an empirical (user tests) and analytical (heuristic evaluation with Web heuristics) evaluation method to identify a greater number of usability issues in learning materials elaborated with and without the cognitive strategies. During the usability evaluations, the focus was the items related to the organization and structure of the content by the interface.

The method used to prove Hypothesis 2 is the observation of where the selected cognitive strategies were applied to solve the recurring problems during the design of the learning material. This observation was planned with the support of a pattern language [12], which captures the best practices of the patterns' identification and writing process.

The patterns of this pattern language provide directives to understand the concepts of patterns and pattern languages, and means of structuring the patterns through the elements that compose the presentation form. This is useful in tasks such as identifying the pattern's subject, as well as the problem solved by the pattern, and indentifying the invariance of its solution, besides directives that must be considered in naming the patterns and making them more comprehensible to readers.

The dynamics used in conducting the case studies are:

  1. The teacher creates the content that is made available as learning material (a hyper document), using Web knowledge and their teaching strategies (referred to as intuitive strategies).
  2. A specialist in the selected cognitive strategies creates new learning material (a hyper document), starting from the material created by the teacher, containing Liebman's strategies, but keeping the original content elaborated by the teacher.
  3. A pattern specialist (HCI and pedagogical patterns) creates a spreadsheet detailing where and what kind of cognitive strategies may be applied in the learning content and the comments made by the cognitive strategy specialist.
  4. The material designed by the teacher and the material designed by the specialist in cognitive strategies is evaluated by two distinct groups of specialists in usability evaluation that perform the heuristic evaluation of this material, with the purpose of finding usability issues.
  5. After the heuristic evaluation, the material created by the teacher and the material created by the specialist is made available to two distinct student groups during the user tests stage, in which two distinct evaluator groups observe the interaction of those users with the designed interface for the purpose of finding other usability issues. The evaluator groups then pick out testimonies from these users during their interaction with the designed material.

Proceeding with the previously mentioned steps, four sets of learning material were created and evaluated in two case studies.

[Continue to page 2]

 

Cog-Learn: An e-Learning Pattern Language for Web-based Learning Design
 

August 4, 2009

Page 2 of 4

[previous page]


Case Study 1. Case study 1 is composed of Learning Material 1.1 (M1.1), designed and intuitively organized by a teacher, and Learning Material 1.2 (M1.2), designed by a specialist in selected cognitive strategies. Both materials have information about material resources management in health institutions and were studied by students from the nursing department at Universidade Federal de São Carlos, Brazil (UFSCar).

A pattern specialist participated in the design of M1.2, pointing out the interface locations where the cognitive strategies specialist would insert a cognitive strategy, and asking why it was decided to insert such a strategy.

After the conclusion of M1.1 and M1.2, the heuristic evaluation phase began. We ended up with five reports for each material containing usability issues and one general report for each material, which compiled all the issues found by the evaluators according to the heuristic evaluation method.

In the user tests, five evaluators observed the interaction of two student groups that studied M1.1 and M1.2 separately. Each evaluator observed how the students interacted with the material and, after concluding the tests, a joint report was created that collected all the results. At the end of each interaction, each student filled out a questionnaire detailing the degree of satisfaction he or she experienced while using the material.

Case Study 2. In case study 2, two sets of learning material (M2.1 and M2.2) were also generated. M2.1 was designed and intuitively organized by a teacher, while M2.2 was designed by a specialist in the selected cognitive strategies. Both materials were about heuristic evaluation and were studied by computing students at UFSCar.

The pattern specialist participated in the design of M1.2, pointing out the interface locations where the cognitive strategies specialist decided to insert a cognitive strategy in the M1.2 design, and asking the why those inserts were chosen.

In the heuristic evaluation and user tests phases (step 5), the same procedures used in case study 1 were followed.

Results: Pattern Formalization
After completing the two studies, we tabulated and analyzed the results that were collected from the usability evaluation. Using heuristic evaluation, we looked at both the quantity of issues that were recorded, as well as their severity.

From the "Usability Guidelines for the Content Elaboration for Web," we collected the quantity of guidelines that were identified as "present" in the learning materials analyzed. In the user tests, we used the "Software Usability Measurement Inventory" (SUMI) questionnaire to note the time each student took to study the learning materials, the quantity and severity of issues found, average time to find concepts in the material, and user satisfaction.

The results collected were documented at great length [14]; a summary of the results follows:

 

Cog-Learn: An e-Learning Pattern Language for Web-based Learning Design
By Junia Coutinho Anacleto, Americo Talarico Neto, and Vania Paula de Almeida Neris, Federal University of Sao Carlos, Brazil

August 4, 2009
Page 4 of 4

[previous page]

Potentiality and Restrictions of the Proposed Pattern Language
We emphasize the possibility of inserting new patterns or substituting some existent pattern for another in the proposed pattern language. In the first abstraction level (Figure 3), which treats the pattern planning tasks, a new specialization can be introduced under the pattern active student, which treats other types of learning—collaborative learning, learning by doing, and learning by observing. Notice that collaborative learning was introduced in this pattern language, but it was not explained in detail because it is not within the scope of this work.

The data collected in the case studies was insufficient to think of more specific patterns for this pattern language. In the knowledge retention pattern [18], the learning material must try to integrate the new material with information previously presented through comparisons that reference new and old ideas, considerations, tables, conclusions, and exercises. The case studies provide sufficient data only to specify the knowledge retention pattern through the knowledge retention types. In future research the knowledge retention types pattern should include a detailed description about the motivation of using each one of knowledge retention items (considerations, tables, conclusions and exercises) and show examples.

The groups and subgroups of patterns also can be replaced or extended according to the need of each teacher. Group 3, which includes HCI patterns (Figure 5), could be replaced by another collection of patterns of the same context or it can be extended by patterns that discuss particular details, for instance, the colors issue treated superficially in this work due to the lack of patterns in the literature that address this theme.

The pattern language to support e-learning is not totally formalized, but that is not necessarily a problem, once it is known that one of the main pattern characteristics is the evolution from projects obtained in practice.

For this reason, we believe that the relationship and the detection of the lack of a pattern in this language also can be obtained from practice, as accomplished in the case studies.

Cognitor: A Pattern-Based Tool to Support the Design of Learning Material
Cognitor is a tool developed to assist teachers in their task of designing and editing high-quality instructional material for e-learning. The instructional material produced by Cognitor can be reused in other contexts of learning, therefore it is designed following the learning object concepts, and can be executed in many e-learning platforms, and therefore it can be exported in SCORM and HTML.

These two characteristics are basic for competition in the market of distance courses production and advantageous for the student and for the teacher. The advantage for students is that when the learning objects are well chosen, it can help their learning process. It can benefit teachers because they have available a great amount of learning objects. Thus they can plan their lessons making use of these objects, obtaining better flexibility to adapt to the rhythm and the interests of the students.

Beyond the previously mentioned advantages, Cognitor also offers aid by means of the computational representation of the Cog-Learn, expressed as functionalities to the teacher. These functionalities let teachers design and edit their instructional materials, using the success solutions for recurring problems that have been identified, widely used in real projects, and finally registered by specialists in the format of a common language of easy access for the ones involved in the e-learning project.

The instructional material is designed considering the implicit questions of the Cog-Learn patterns such as: the design of the lesson structure, as well as of its content; the project to elaborate the sequence of actions; the aid during the accomplishment of the course with the stimulation of the student's cognitive strategies; issues on interaction design (usability, accessibility, navigation and layout); and finally, considering topics of portability, content reuse and performance control by means of the development of contents according to SCORM.

These characteristics had provided the idea to use the Cog-Learn patterns as a framework to support the design and the edition of instructional material for e-learning, considering the pedagogical questions and the questions relative to the student's interaction with the interface.

The idea of expressing the Cog-Learn as a framework comes automatically, as well as in other domains, like software engineering, where the design patterns demonstrated to be so useful that many of them were included as a framework of tools that support the coding in certain programming languages, after standing out in the arsenal of tools and techniques for software development [9].

Cognitor (Figure 7) also offers the functionalities of design and reuse of learning objects by means of: an HTML publisher, a media aggregator, and the support of the pedagogical and HCI Cog-Learn pattern.

Figure 7

Figure 7. Cognitor's main interface is shown.

To create learning material, teachers may choose one document organization previously defined, create a new document organization, or use a pattern to help him in this task of planning the material. Afterward, the teacher can edit each page using the media insertion or the interaction design area. The interaction design area provides functionalities about either pedagogical or usability design issues like course design, content structure, navigation, layout, and learning activities.

One example of a pattern that is included as functionality of Cognitor is the knowledge view pattern [18], which supports the automatic generation of the content structure of the learning material by using the concept map theory. The teacher need only provide the Cognitor with the concepts, the links between them, and the names of these links. Cognitor builds the content structure and creates the pages and the links between them. So Cognitor provides useful information about the links between the related concepts previously defined by the teacher on each page generated with the pattern's aid.

After the content insertion, the designed material can be exported in HTML for execution in a Web browser, or in a SCORM format, with learning objects and metadata, for execution in a learning management system (LMS).

Cognitor also offers an innovative technique that considers the common sense knowledge provided by the Brazilian Common Sense project [Open Mind Common Sense Brazil (OMCS-Brazil)]. The aim of this project is to provide computers with knowledge and to promote natural interaction between the humans and the computer.

We decided to develop a common sense module to help the teacher when using the knowledge view pattern, which proposes the planning of the lesson using a conceptual map. When identifying concepts, teachers can query the common sense knowledge base to find related concepts, and include it in the conceptual map. This is made using a Wizard that guides the teacher.

Figure 8 shows instructional material designed using Cognitor and delivered in a Web LMS platform. The characteristics of this material are 1) it's composed by Assests and SCO's, described in an XML manifest in conformance with SCORM, and 2) its interface is implicitly designed using the design patterns of the Cog-Learn pattern language.

Some of the options offered by the Cognitor and the Cog-Learn pattern language were discussed here, but other ideas, as well as new patterns, will also be analyzed and incorporated in the other versions of the prototype tool, focusing on the quality and usability of designed material.

Figure 8

Figure 8. Cognitor also designs Web instructional material.

Conclusions
In this paper we have presented Cog-Learn, a pattern language for e-learning support which contains HCI patterns, pedagogical patterns, and the ones based on cognitive strategies, aiming at supporting the teacher in the task of designing Web-based learning material.

Such pattern language comes from the study of cognitive strategies application used by Liebman [11] and extended in a way to support teachers in the task of designing high-quality learning material for e-learning. The case studies provided the selection of patterns from the literature and the writing of new ones, presented here. They also allowed us to verify that cognitive strategies increase the usability of learning material for e-learning and, consequently, its quality.

Furthermore, we also present the Cognitor, a computer-based tool, and more specifically, a pattern-based editor that incorporates the Cog-Learn pattern language to support teachers in their task of designing learning material that promotes active learning, reducing knowledge-acquisition complexity.

In the future we intend to perform usability evaluations of the Cognitor tool in order to find out if it could be successfully used to promote a quick and efficient integration between the different professionals involved in the teaching and learning process in e-learning environments.

It is important that researchers and developers involved with e-learning realize that e-learning environments must be adaptable to teachers and students of varied fields, and that pedagogical support in the creation of material is needed, as well as targeting questions about the quality of the learning material designed. It is expected that this work will contribute to this objective.

References
1. Alexander, C. et al. A Pattern Language. Oxford University Press, New York, 1977.

2. Ausubel, D. P. Educational Psychology: A Cognitive View. New York: Holt, Rinehart and Winston, 1968.

3. Beckman, P. Strategy Instruction. ERIC Clearinghouse on Disabilities and Gifted Education Arlington. Educational Resources Information Center, 2002.
http://www.ericfacility.net/databases/ERIC_Digests/ed4743....

4. Bergin, J. "A Pattern Language for Course Development in Computer Science." Pace University, 2002.
http://csis.pace.edu/~bergin/patterns/coursepatternlangua....

5. Borchers, J. A Pattern Approach to Interaction Design. John Wiley & Sons, 2001.

6. Fidel, R. "Qualitative Methods in Information Retrieval Research." Library and Information Science Research, 15(3), 1993, pp. 219-247.

7. Fricke, A. and Völter, M. "Seminars—A Pedagogical Pattern Language about teaching seminars effectively." Proceedings of EuroPLoP, Germany, July 10, 2000.

8. Gagné, R. M. The Conditions of Learning, 3rd Edition. Holt McDougal, 1974. 9. Gamma, E., Helm, R., Johnson, R. and Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

10. Kessler, G., Rosenblad, K., and Shepard S. "The Web can be suitable for learning," Computer, 32(2), 1999, pp. 114-115.

11. Liebman, J. "Teaching Operations Research: Lessons from Cognitive Psychology." Interfaces, 28(2), April 1998, pp. 104-110.

12. Meszaros G. and Doble J. "Metapatterns: A Pattern Language for Pattern Writing," in Proceedings of the Conference on Pattern Language of Programming, Illinois, Sept. 4-6, 1996.

13. Montero, F., Lozano, M., Gonzáles, P. and Ramos, I. "A first approach to design web sites by using patterns," in Proceedings of VikingPLoP Conference, 2002.

14. Neris, V.P.A, Talarico Neto, A, Silva, J.C.A, and Mascarenhas, S.H.Z. "Hyper Documents with Quality for Distance Learning: Cognitive Strategies to Help Teachers in the Navigational Project and Content Organization," in Proceedings of the 11th Brazilian Symposium on Multimedia and the Web, Poços de Caldas, 2005.

15. Nielsen, J. Usability Engineering. Academic Press, Cambridge, 1994.

16. The Pedagogical Patterns Project, Retrieved 2001: http://www.pedagogicalpatterns.org.

17. Preece, J., Rogers, Y., and Sharp, E. Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, 2002.

18. Talarico, N. A.; Silva, J.C.A.; Almeida, V.P. "Padrões para Apoiar o Projeto de Material Instrucional para EAD," in Latin American Conference on Pattern Languages of Programming-SugarLoafPLoP, 2005. /in Portuguese/.

19. Tidwell, J. "User Interface Patterns and Techniques": http://time-tripper.com/uipatterns. 2003.

20. van Welie, M. Patterns in Interaction Design, 2003: http://www.welie.com.

21. West, C. K., Farmer, J. A., and Wolff, P. M. Instructional Design: Implications from Cognitive Science. Allyn and Bacon, 1991.

19:39 | Lien permanent | Commentaires (0) | |  del.icio.us | | Digg! Digg |  Facebook | | | |  Imprimer