Crina Damsa, Patrick Sins, Jerry Andriessen and Gijsbert Erkens
UU Universiteit Utrecht, The Netherlands
November 13, 2006

Course name: Educational and instructional design

Institution: Utrecht University, Faculty of Social Sciences, Department of Educational Sciences

Course instructors: Gijsbert Erkens, Sandy Werdmüller–Von Elgg

Scope: Past practices and experiences

No. of participants: 20 students in autumn 2006, aprox. 80 students in spring 2007

Target populations: Second year bachelor students and pre-master students in Educational Sciences

Content areas/Disciplines: Educational design, Design methodology, Process management

Duration of the course: 20 weeks (11^th of September – 10^th of November 2006 / 5^th of February – 20^th of April 2007)

No. of instructional hours: 420

Additional background information
The course is a second-year bachelor course at the Department of Educational Sciences, Faculty of Social and Behavioural Sciences. The Department offers a bachelor study of three years in total which can be completed by a one year professional master, or a two year research master in Educational Sciences. A qualification in Educational Sciences offers the graduates a large range of job perspectives in: educational research, the field of teaching and education (e.g., curriculum design, coaching and -supervising, teaching in teacher training institutions), private companies (e.g., instructional design, consultancy, training), or in government positions (e.g., educational policy, consultancy).

An important aspect in the design of this course is that students collaborate intensively in small groups, since we expect that it stimulates students to be more aware of and reflect on their learning process.

Course description:
This course is one in a series of instructional design courses offered during the first two bachelor years. By attending this series of courses students gradually build their knowledge and improve their skills in instructional design. The participants in this course consist of students completing their bachelor or pre-master study in Educational Sciences, and of students that attend this course as a part of their minor. Normally, the total group of students consists of more than 80 persons. For this scenario, however a distinct target population participates in the course, namely a group of approximately 20 part-time students in Educational Sciences.

Goals
The goals of the course reflect the principles of trialogical learning in various degrees. For instance, the course aims to make students familiar with the major theories and methods of educational and instructional design and allows them to apply this knowledge in new learning situations. In addition, students are presented with an authentic instructional design task, in the sense that they work for a real customer in a project-based work set-up. In this context students are confronted with realistic, complex problems, which they have to solve by working together. The results of this process take the form of new (reusable) artefacts, which reflect the students’ understanding of instructional design theory and of educational practices. Ultimately, the course intends to support students to gain a elaborate insight in the possibilities for implementing their design product in practice.

Set-up
The course approaches the design process from a perspective, in which traditional instructional design theories are combined with socio-constructivist ideas. This means that students will address the design process as a step by step activity which is integrated into a professional and social context.

The course has a complex set-up, which includes a project-based activity component and a lecturing and seminars component. The lecturing and seminar component is organized in two topical variants: one concentrates on the instructional design within the private sector (variant B) and the other focuses on characteristics of the process within the educational field (variant C). Both variants are completed by a general part (part A), which offers students the opportunity to discuss the general theoretical framework of the instructional design process and any other theoretical insights related to the design process. Students may associate their choice for a certain type of design task with one of these variations.

After making their choice for a particular variant of the course (instructional design in education or in the private sector) the students make teams. Each team is required to find a customer and to agree upon a design task. The work of the teams on the artefacts represents the essence of the course. During their activities students are scaffolded in their learning process.

Video conference lectures: are planned at certain times, with the purpose of sketching a theoretical framework, based on the scientific literature. For this purpose the teacher will have a online video conferencing tool at his disposal.

The seminars are face-to-face meeting, organized every 3 weeks, wherein each team has the opportunity to discuss their design or any other issues they consider important with their supervisor, and to ask questions when parts of the theory are unclear.

Performing the design task requires from students: regular face-to-face or virtual meetings with the team, planning (shared) tasks, consultation on regular bases, and field work in the form of interviews with subject matter experts. Students’ collaboration during most of these activities will be facilitated using electronic tools.

Work in this course is organized according to the principles of project-based education. This involves applying the knowledge and skills learned and exercised in a realistic and practice-oriented learning environment. The principles of project-based work are translated in the design of the course as follows:

1. An assignment (in the form of an problem-solving task): An instruction problem

2. A customer: An external instance

3. An independent project team: Maximum 4 students

4. A product: A learning environment/onderwijsleerpakket

5. A process: A design method

6. Quality criteria: An explanatory report and evaluation

7. A process supervisor: The teacher of the course

Set-up/Narrative

1. PROJECT INITIATION

Task 1.1 Form groups

The project initiation takes place at the beginning of the course. Students have a course manual at their disposal and receive explanations from the course coordinator about the course. All students enrolled for this course meet once a week for a lecture. In the case of this course the group of students meets once at the beginning of the course, because the lectures will mainly consist of online video-conferences.

During this phase students are expected to form a team (of three students preferably) by themselves and start to look for a customer.

Task 1.2

Examples of customers are: schools, private companies, governmental or non-governmental institutions, etc. In case teams can not find a customer the teachers can offer help. The coordinator has a database with potential customers, so he/ she can offer the necessary support. Teachers also have design tasks prepared, in the rare case teams do not succeed in finding a real customer. After finding a customer teams are expected to make an initial agreement about the design task.

2. PROJECT PLANNING

Task 2.1: Write a project proposal
The planning of the design activities and the division of tasks and responsibilities within the team represent significant moments in performing the design task. These activities take place either during face-to-face or virtual team meetings. The memos of this face-to-face meeting are uploaded and placed in a shared map online and can be used as input for further activities.

The consultation and the planning can also take place during virtual discussions through a computer-supported (a)synchronous tool. This communication tool must offer the possibility to save its history and it should be accessible every time it is needed.

In the virtual planning tool the ‘milestones and deliverable’ can be established, as well as the responsibility for each team member.

The preparation phase also includes writing a project proposal for the customer, wherein the team stipulates the necessary conditions of the design process. In this proposal the agreement reached in the previous discussions will be worked out in more detail.

The aspects that are to be outlined in this plan are:

• Purpose of the design process;
• Work plan (tasks and responsibilities);
•  Milestones and deliverables;
• Budget (hypothetically, because normally the students do not get financial rewards); and
• Schedule.

This is a collaborative activity in which the team members will work together, synchronously/asynchronously, on writing, editing and revising the draft of the project proposal.

3. EXECUTION AND CONTROL

In the process of instructional design, three phases can be distinguished: analysis, design, and evaluation. In this course the phases of the design process are combined in a complex task. The concrete form of the product can vary, depending on the demand of the customer, from workbooks to software for a computer supported learning environment, from on-the-job learning activities to simulation games, from concrete study material to an evaluation procedure, etc. The teams have to organize their design activities according to the successive phases described above.

Task 3.1: Learning needs analysis/goal analysis/performance assessment

A learning needs analysis in the target group is done with purpose of identifying the specific learning needs of the target population. The analysis can also take the form of a goal analysis or of a performance assessment. The conclusion of this analysis results in a specification of an instructional problem, which will represent the starting-point in the design process. This phase consists of the following activities:

Document analysis. Using the document management system, teams make an inventory of information from the customers’ documentation. The searching activities may take place individually, but every team member stores the information in a common online folder. This file is uploaded in the shared space and visible to all team members.

Interviews with person(s) inside the customer institution. Interviews are held with the person(s) responsible with the organization of education or instruction within the respective institution. Developing a questionnaire or a list with open questions concerning the learning needs of the target group will help students to conduct the interview in an efficient manner. The questionnaire is developed in collaboration, using a brainstorming tool that helps students to identify topics to be worked out in the questionnaire. The questions can be accessed and revised by all team members.

The processing notes from the document analysis and outcomes of the interview result in the writing of a draft for the needs analysis partial report.This is a collaborative activity with all team members taking part. The needed relevant files (e.g., notes, interview texts) are placed in the shared space are visible for everyone. A concept-map tool/graph tool is used to organize the topics mentioned in the notes and interviews.  Furthermore, the team members can work together, synchronously/asynchronously, on writing, editing and revising the first draft of the learning needs partial report. A chat tool may support communication during these activities.

Task 3.2: Domain analysis (task or cognitive analysis)

A domain analysis is conducted based on the instructional problem identified in the learning needs analysis. Based on the conclusion of that analysis the content of the instruction (i.e., knowledge, skills, procedures, and attitude) must be determined. Depending on the type of design product required by the customer a task (procedural) of cognitive (topic) analysis is conducted, comprising the following activities:
Interviews with subject matter expert(s) (SME). The interviews are held with an expert within the respective occupational field (educational or professional), preferably from inside the customer institution. Developing a list with (open) questions concerning the chosen topic will help students to conduct the interview in efficient manner. The questions are developed in collaboration with others, using a concept-mapping tool which helps students to create an overview of topics to be worked out in the questionnaire. The questions can be accessed and revised by all team members.

Documents analysis. In case the interview with the SME in not possible or if it provides insufficient information, a document analysis can complement the picture. Documents containing function profiles, task description or assessment procedures can offer information on topics that must be included in the design product. Team member can split task when analyzing the documents, but all the information is saved in the common folder (shared space).

The processing notes from documents analysis and the outcomes of the interview results in the writing of a draft for the task or cognitive analysis report. This is a collaborative activity with all team members taking part, which takes place either face-to-face or online. If chosen for the second alternative, the relevant files can be placed in the shared space and can be accessed by everyone. A concept-map tool/graph tool is used to organize the topics mentioned in the notes and interview outcomes.  The contextual and learners’ analysis (only if necessary) completes the domain analysis. The report on this analysis emphasizes the characteristics of the physical environment where learning will take place and the characteristics of the target group.

The results of the domain analysis are presented as a part of the domain analysis report.

Task 3.3: Designing a macro and a micro design product

The creation of the design product is the ultimate goal of the process. Two different products will be delivered: a macro-design product, which is represented by the instructional objectives, a sequencing of the topical content according to these objectives and a global outline of the course (i.e., tasks, activities, materials, evaluation methods); a micro-design product, which is represented by one unit of instruction (i.e., lesson, training part, task in an electronic learning environment) and the concrete tasks, activities, learning materials, and evaluation form. The design phase consists of the following activities:

Specifying the instructional objectives has the purpose to concretize end terms of this design activity. This is a collaborative activity, wherein all team members participate in formulating the end terms in a concrete and measurable form. The instructional design theory offers students the criteria based on which the objectives for the respective design must be formulated. The theory also offers concrete models that can be used to formulate these objectives correctly. Students must choose for the adequate type of objective and the proper formulation based on this theoretical information and the results of their previous analysis (on learning needs, task or cognitive analysis). Making this choice involves discussion, use of arguments and maybe visualisation of these arguments. Every student participates actively in the discussion and offer arguments for an expressed opinion.

Students can use the text processor to work on different drafts of the objectives and use the chat tool/discussion forum for further consultations on the matter.

The sequencing of the topics that will constitute the content of the instruction, takes place based on the outcomes of the domain analysis and the instructional objectives. This activity involves all the member of the team and can be supported by a concept-map/graph tool, chat tool and a shared text editor.

Creating an outline of the macro design-product requires intensive collaboration between team members. A separate component of the outline must be created (i.e., tasks, materials, evaluation methods) but they all must form a coherent instruction. Therefore, the team members work together during face-to-face meetings and in the shared space specifying these components of the design-product. It is also possible that team divide tasks, but their work has to be regularly available for feedback by their fellow team mates. The micro-design product is a component of the macro-design and can take the form of a lesson, a training part, a specific tool, or an exercise series. It involves again intensive collaboration between the team members, who must first agree on which part of the macro-design product to be worked out in detail and than create all the concrete components of it.

Task 3.4: Designing an evaluation instrument

The design product must be evaluated in relation to the end terms, which are the formulated instructional objectives or expected work behaviour. For this purpose an evaluation instrument must be designed. First of all, the type of evaluation and the adequate evaluation instrument must be developed or selected based on the theoretical framework students have chosen. Subsequently, the team must design the evaluation instrument and, if possible, also conduct an evaluation of their design product.

Task 3.5: Writing the explanatory report

In parallel with designing the product, students must also work on their explanatory report. In this report each team must explain their choice for specific analysis and design methods and justify their choice using theory-based arguments. This report requires the collaboration of all members of the team.

In writing this report all artefacts students have produced can be used. This means that the partial reports, the notes, discussions and the theory presented and discussed in the lectures and seminars can be used as input for the report. 

Activities required from team members are:

• Concept-mapping of the theoretical content;
• Making process notes and memo’s;
• Redraft partial reports, successively introduce new concepts; 
•  Annotate and revise text;
• Upload new information;
• Consultation and discussion.

4. DELIVERY

First, students deliver their artefact to the customer. The deliverable is the design product, together with the evaluation instrument.
Second, in order to round off the course team must present a number of deliverables to the teacher: a) the design product, b) the evaluation instrument, and c) the explanatory report.
For getting the deliverable ready teams must:

• Collect all the critical artefacts in a shared map;
• Revise the texts and agree on the lay-out;
• Send in the deliverables.

OTHER COURSE ACTIVITIES

• During the lectures and seminar, students have the opportunity to discuss theoretical issues with the teacher. A discussion forum can offer additional support through asynchronous discussions on theoretical matters. Students can discuss the theory presented in the lectures; ask questions related to these theoretical matters, and share summaries or notes on the literature with their fellow students. A chat tool is also available for synchronous discussions.
• The teachers are also available for face-to-face feedback sessions and discussion with the groups. As mentioned in the scenario weekly meetings/feedback sessions are planned. Teams sent their draft artefacts in advance, so the teachers have time to prepare and read the material. During these meetings, teachers give teams feedback on their artefacts and answer students’ questions.
• Face-to-face meetings between members of the team. In these meetings the strategies, responsibilities, dead-lines, and other tasks of logistic nature are discussed.
• Reflection on students’ activities will take place during collaborative activities inside the teams. The workload for the students will be in this case less than in when they are asked to write a complete reflection report.

Critical features/Requirements

Non-technical

• Course set-up (project-based work) must be made clear to the students (e.g., in a course manual);
• Teacher must provide consequent feedback on students’ artefacts;
• The customers should provide the teams with authentic assignments;
• Clear communication of the expected outcome and purpose of the assignment;
• Transparent structuring of the different phases and briefing on the tools to be used (i.e., showing them the usage of blog, forum and wiki by using an example);
• The subject-matter expert must be available for providing information.

Technical

• An integrated tool which provides support for collaborative project-based work;
• The system must provide support for collaborative writing work: text editor, shared database;
• The system must provide support for communication between team members (chat, discussion forum/wiki) and for process management;
• System should offer the possibilty to check the history of the task;
• System executable by different browsers and operating systems;
• All versions of the artefacts must be ‘visible’ : status and visual form of representation;
• Support of private areas and shared areas should be present; with an overview of who is active at the moment;
• Possibility to save chat and forum discussions/wiki sessions as artefacts.

Rationale

Knowledge creation. The knowledge creation and advancement perspective is reflected at the product as well as at the process level. The artefacts are the result of knowledge creation processes. For example, the learning needs analysis report offers a new insight in the customer institutions’ learning needs; a design product is a combination of prior and new knowledge and represents an advancement regarding training within the respective professional field. Knowledge creation features can be identified also in the design process and other related activities. For instance, while performing a design task students create knowledge concerning the design mechanisms; while striving to deliver a good product they also improve their knowledge of quality norms of instructional design.

Collaboration. This courses’ set-up is meant to support and nurture collaborative activities. From the start of the project teams are required to collaborate in finding a customer. Collaborative efforts are further required when making decisions, conducting analysis, processing data, when reporting on the collected data or when brainstorming with the purpose of constructing the design product. Collaborative work on the shared artefacts will not only result in the artefact itself, but also in the advancement of student’s skills and knowledge on collaboration for instance (meta-skills).

Work around shared objects. The set-up offers perfect opportunities for the team members to work together on shared artefacts. Most of the artefacts are developed in collaboration (i.e., design products, learning needs analysis, evaluation instrument). A good example is designing the content structure for the macro-design. Using information provided by the needs analysis report, the topic analysis report and the learning objectives, team members can brainstorm and create a first draft structure of the design. During the intensive collaboration, this structure progresses in a dynamic fashion. The various versions of the design product can be considered as intermediate artefacts.

Generating artefacts for authentic purposes. During this course the team meets with real customers and experts from the professional field. The assignments given by the customers are authentic assignment and the designed products (e.g., courses, trainings, learning books, and e-learning  tools) are in most cases used in practice. The quality of the design product is assessed by the customer  based on market criteria, which are also used when designs are produced by a professional educational designer. In this way, students experience the demand of the professional field and develop accountability, especially when knowing that their design products will be implemented in practice.

Research hypotheses

Research objectives

• Gain insight in the conceptual foundations of trialogical learning processes;
• Capture the developments in educational practices as a result of the implementation of KP-lab learning principles and technology;
• Develop useful analysis tools that allow monitoring of trialogical learning on the basis of written artefacts;
• Collect information about how the trialogical tools support the knowledge advancement process;
• Identify the impact of the implementation of trialogical learning principles at pedagogical and organizational level.

Related tools, artefacts, and activities:

Tools

Shared document management system/database: offers the possibility to store and manage documents and different types of information (e.g., text, audio, video);

Synchronous/asynchronous text editor: gives students the opportunity to work together at the same text

Chat tool: supports synchronous discussion at two different levels, team level and group (class) level;

Web-based discussion forum/wiki: supports asynchronous discussions at group (class) level;

Concept-map tool: brainstorming instrument necessary when designing data-collection instruments (for the learning needs and topic analysis report), when thinking about the form and structure of the design;

Graph tool: offers support for visual representation of theoretical or non-theoretical topics, of argumentative discussions

Video-conferencing tool: offers the opportunity to organize digital lectures;

Mobile devices for data-collection (video and/or audio recorders, preferably digital): possibility to record interviews and discussions with SMEs and ideally automatically transfer them to a text editing tool;

Pda devices: possibility to make notes while working on the field, mobile information search device and mobile communication device.

Artefacts

Critical artefacts:

•  project proposal
• learning needs analysis report
•  domain analysis report
• design product (learning environment (software/learning materials/course/training):
• macro-design product: instructional objectives and global outline of the course (units of instruction, tasks, materials); 
• micro-design product: - outline of a unit of instruction (tasks, materials, activities)
• evaluation instrument
• explanatory report

Other artefacts:

• contextual analysis
•  learner and contextual analysis
• digital drafts and final version of the explanatory report
•  meeting memo’s
• data collected from interviews
• concepts maps
• synchronous and asynchronous discussion files
• discussion files (in the discussion forum)
• reflective notes.

Activities related to the process:

• collect data on the field
• process collected data
• write and edit texts
• conduct discussions on various topics (organizing work, content topics, procedures)
• conduct interviews

Activities related to the people:

• work in collaboration with the team mates
• reflect on own activities and on collaborative activities
• analyse own knowledge advancement.

Experiences

The current course is running well, the students are very involved, enthusiastic and active. There is a need for an integrated tool that offers more appropriate support for collaborative activities.