Program design and implementation (CAS HEiE) - week 1

Jump to navigation Jump to search
Program design and implementation (CAS HEiE)
2023/03/20
Other module pages




Week 1 - Foundations of program design and implementation

  • Interactive Class: Monday March 13: 17h00 - 19h15 (short break in the middle)
  • Interactive Class: Wednesday March 15: 17h00 - 19h10 (short break in the middle)
  • Public tutoring session: Friday March 17: 17h-19h (17h-18h if low interest)

Learning goals of week 1

  1. Understand the structure, learning environments, and requirement of this module 3 (-> Program design and implementation (CAS HEiE))
  2. Recall general program design, program implementation and quality assurance principles
  3. List steps of educational program design
  4. Use the ADDIE instructional design method in a user-centered interpretation
  5. Identify and discuss items in the educational program description grid
  6. Start using the educational program description grid (and make adaptions)
  7. Familiarize with Moodle (as teacher/course designer)

Introduction to module 3

Presentation of the module "program design and implementation"

  1. Learning goals, see also Program design and implementation (CAS HEiE)
  2. Weekly organisation

Assignments, learner support and evaluation

  1. Learner support (public tutoring session, forum)
  2. Assignments - continuous work on the same design document / design of Moodle activities / peer commenting
  3. Final project
  4. Reuse of module 2 work is encouraged
  5. Course, policies and learner assessment see also Program design and implementation (CAS HEiE)
  6. Grading rubrics (presented on week two, can be negotiated)
  7. Group work (slight extra requirements)
  8. Individual work (peer commenting)

Learning environments

We will use different platforms:

  1. Learning Management platform: Submission of assignments, extra materials : Moodle UniGE
  2. Moodle to learn Moodle: Please log in to https://arbores.tech/lms/ and change your password. The instructor will display the file (module_3_participants.csv) with login + password information. It is available in the Moodle UniGE page. The system will then ask you to change the password.
  3. Arbores Tech Wiki (this wiki, lecture notes and description of learning activities, reading only): https://w.arbores.tech/wiki/cheie
  4. Various other content platforms and online services, e.g. Google Docs

Reading

We will use three main textbooks, plus various extra materials of all sorts (academic, research, tech manuals, advice, ...). All reading is optional unless otherwise specified.

  • Bates, A.W. (Tony) (2022). Teaching in a Digital Age: Third Edition, Vancouver, B.C.: Tony Bates Associates Ltd. Retrieved from https://pressbooks.bccampus.ca/teachinginadigitalagev3m/ You also can download this book in various formats, e.g. PDF, Print PDF, EPUB, HTMLBook. Make sure to get the official final version, since draft versions may have different page numbers.

This week (week 1)

Monday:

  • General introduction to program design and implementation
  • Innovation and change management
  • Project management and quality approaches (if time)

Wednesday:

  • Instructional program design
  • Checklist for educational program design and implementation
  • Assignments

What is program design and implementation ?

Program design and implementation has several facets. There are different ways to look at program design and implementation. Important distinctions can be made between:

  • social theories that explain how implementation happens and that define how they should be studied,
  • general design and implementation principles and methods,
  • specific principles & methods that can be applied to educational program and course design (main focus of this course module)

Descriptive program implementation models

  • General description implementation models usually stress some kind of circularity. They describe implementation processes as seen by researchers. Intervening factors shape design explicitly or implicitly.
Program implementation loop

Scope of implementation models differ. Some start from goal definition, others from problem definition, still others from the emergence of problem perception.

Applied Implementation and Evaluation studies

  • Depending on their scope and aims, implementation and evaluation studies do not lead to the same type of results.
  • Restricted evaluation studies may show that promised outputs were done, that some outputs did happen, but not necessarily why. They also avoid looking at non-indented outcomes (whether positive or negative).
  • Evaluation is discussed in the last course of this CAS.
Scope of implementation research model
Discussion:
  • How do we know if we pursue the right goals (or aven that we have clear goals) ?
  • Should we look at global impact (intended or non) or focus on measurable outputs ?

Proscriptive models

Proscriptive models define how programs should be designed and implemented. We will lock a few principles of interest to educational program design.

  • See also project management below

Pretotyping

Pretotypes occupy the space between abstract ideas and prototypes. (Alberto Savoia, 2022 [1]). Often, programs are launched without really understanding what and if "it" is needed, pretotyping can help clarifying purposes. According to Savoia (p. 12), (ex-Goggle management):

  • The Right - It is an idea for a new product that—if competently executed—will succeed in the market.
  • The Wrong - It is an idea for a new product that—even if competently executed—will fail in the market.

User-centered design

User-centered design is a broad family of design methods that engage participants in the design. Participatory design is a popular variant.

Participatory design

The two figues below summarize the principle: design is circular and participants are engaged in each cycle.

Adapted from Hakansson (dead web page)
Preece, Rogers et Sharp[2]

Participatory design is rarely carried out in its pure form. "Waterfall" design elements ("top down implementation without active user participation and limited iterations remain popular.

User Centered Design (USD) cycle according to ISO

User-centered and participatory design focus a lot on usability, usefulness and satisfaction.

According to David Travis, in 13407 is dead. Long live ISO 9241-210! (June 2011, retrieved June 6 2011), ISO 9241-210 (formerly ISO 13407) defines 6 key principles for user centred design:

  1. The design is based upon an explicit understanding of users, tasks and environments.
  2. Users are involved throughout design and development.
  3. The design is driven and refined by user-centred evaluation.
  4. The process is iterative.
  5. The design addresses the whole user experience.
  6. The design team includes multidisciplinary skills and perspectives.

From the same author, free on-line texts about user-centered design and user experience:

Discussion:
  • Did you participate in participatory design in the past ? How did it go ?

Mutual understanding and grounding

  • Mutual understanding among all stakeholders (including future teaching staff and learners) is crucial for the success of a project. Most often, participants in an educational project do not speak the other's "language" and therefore cannot collaborate efficiently. For example, a teacher does not understand an instructional designer, a program manager does not understand basic ICT principles and terms, someone educated in the French system does not understand terms used by AngloSaxon counterparts (even if translated).
  • Mutual grounding also refers to sharing of common beliefs that are carried by the project. Typically, visions and other strategic goals as well as relevant theory.

Aim to identify boundary objects, i.e. objects which are both plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites. [3]. "A boundary object is a ‘thing’ that is both defined enough that several communities can recognise it as the same thing, yet flexible enough that each community can use it according to their own needs." (Matt Cooper Wright, 2012 [4]

Boundary object by Matt Cooper-Wright, Medium, Nov 28, 2012

Basically, a common language that can adapt to different contexts. It can help people crossing boundaries.

Wenger (2000:236) [5] presents three categories of boundary objects.

  1. Artifacts: tools, documents, models shared by Communities of Practice (CoPs).
  2. Discourses: a common language that can be shared across CoPs
  3. Processes: shared processes, routines, procedures that facilitate coordination of and between CoPs
Discussion:
  • Can you remember difficulties of mutual understanding, e.g. between persons with different job backgrounds ?

Context

Understanding cultural context

  • How would you define intercultural competence ? and intercultural intelligence ?

Intercultural competence can be examined through the lens of cognitive, affective, and behavioral (CAB) dimensions. This paradigmatic approach focuses attention on such personal variables as tolerance of ambiguity, open-mindedness, and behavioral flexibility [6]

K. Leung, S. Ang, M.L. Tan, 2014

Leung, Ang & Tan [7] define intercultural effectiveness as being a result of intercultural competence. This component view of what intercultural effectiveness does not explain how it develops. Therefore, another approach is to look (inter)cultural competency in terms of developmental models as well as "intelligence". Cross et al. (1998 [8]), in the context of workplace diversity, laid the foundation of many further studies that have a behavioral and developmental orientation and that also have more practical aims: “Cultural competence is a a set of congruent behaviors, attitudes, and policies that come together in a system, agency, or among professionals and enables that system, agency, or those professionals to work effectively in cross-cultural situations.” Republished as Cross (2012) [9]

Discussion:
  • Do you agree that there could be a difference between cultural competence and cultural intelligence ? If so, what is it ?

Accessibility of online classes and materials

Accès to machines, the Internet, bandwidth, network reliability constraints, special needs, etc.

Discussion:
  • What is your experience in the field ?
  • If already engaged in a program, how to you handle these problems ?

Other

discussion
  • What are other specifics of the emergency context ?
  • How does well-being affect learning ?

The dynamics of Innovation and change

Suggested Reading:

  • McCowan, et al (2022). Enablers of pedagogical change within universities: Evidence from Kenya, Ghana and Botswana. [10] This article address change management problems and strategies, i.e. difficulties of enactment. Quote: “There are certain crucial factors affecting [] implementation: the drivers of the initiative, and the extent to which it is embedded in the structures of the institution and championed by staff members; the presence of a pedagogical vision that is shared across an institution (or on a more modest scale the department or faculty); the necessary resources to implement it in practice, along with the work conditions and incentives required by lecturers; and finally, the opportunities and spaces for teaching staff to engage in reflection on their practice, and transform themselves and their teaching.”
Rogers' bell curve of adoption and Moore's chasm

Diffusion of innovation follows a kind of bell curve.

Rogers (1962) diffusion model includes five stages and accordingly adopter types: innovators, early adopters, early and majority, laggards. Source: Wikipedia.

.... if the chasm can be crossed. Education probably follows a similar kind of pattern, but may require several attemps.

Moore's technology adoption life cycle
The J-Curve pattern

Innovation often has to go through a "J" Curve: After an introduction of an innovation, performance is lower and can lead to a stop. This must be anticipated. Satir, Virginia, et. al. (1991).[11]

J-Curve change model of Satir, redrawn by Philippe Ruffieux
The Process of Transition, JM Fisher 1999/2012, reproduced by BUlletin 2014
Discussion:
  • Obviously, many if not most innovations fail
  • In an educational context, what could be done to minimize the effects of the "chaos" period ?

Bates' Change Management Principles for e-learning

Bates (2000, 1-6) [12] formulates a few principles that change agents should know. Here are few guidelines that an organization could adopt.

1. It's unlikely that new technologies will lead to reduction in spending because investment costs are high and recurrent. However, new technologies may increase pedagogical effectiveness and therefore be profitable. However this requires substantial reorganization of learning and teaching.

2. Technology must be integrated into a wider strategy for teaching and learning

3. “Lone ranger approaches” are useful to get started, but costly and inefficient in the long run.

4. Appropriate technology infrastructure is an essential requirement for technology-based teaching.

5. Faculty members need much more support and encouragement than has been provided to date for their use of technology for teaching and learning”. Effective teaching with technology requires a higher skill level.

6. It is essential that institutions understand the costs of using new technologies.

7. Funding is probably the biggest lever for change.Also, reallocation is the ultimate test of an institution's commitment to teaching with technology.

8. Partnerships and collaboration are strategies for sharing the costs and leveraging the benefits of technology-based teaching.And, an organizational structure encompassing a mix of centralized and decentralized strategies is recommended to support teaching with technology.

9. The implementation of these strategies will require fundamental change in the way our higher education institutions are organized and managed.

10. Change can be a risk. Badly managed organizational projects can lead to disaster.

We can summarize Bates’ guidelines by putting forward the hypothesis that (1) change can only be achieved when the organization as a whole is ready to engage in the process and (2) that change requires good management since there are many perturbations and obstacles.

Project management and quality approaches

Project management methods

There are many ways to organize educational program development.

For larger projects, traditional project management tools can be used. They require definition of elements like:

  • Goals, that define what should be achieved.
  • Work packages that organize who does what (which tasks).
  • Tasks (or activities) that pursue goals and lead to
  • deliverables.
  • Milestones are important completion dates.

Work packages are most often executed by a team and are coordinated by coordinators. Tasks are usually given to a workpackage, but can be shared.

Typically, projects use so-called GANTT charts that include more or less details about tasks, order of tasks, time spent on tasks, milestones, etc. The following chart uses a two-level work breakdown structure (WBS). At the first level, a WBS can be a dimension of the project, e.g. Analysis, Planning, Development, Testing.

Created by Garry L. Booker. It is intended to be more descriptive than the previous example. Retrieved from Wikipedia https://commons.wikimedia.org/wiki/File:GanttChartAnatomy.svg on october 18 21022.

This example shows three kinds of schedule dependencies (in red) and percent complete indications. See also some practical explanation in the Guideline: How to prepare a detailed project plan document. (Wikibook: Development Cooperation Handbook). By the way, some of you could contribute to that free online book.

The Kanban board

We suggest using a light-weight model, e.g. the Kanban board method. It's most simple version looks like this:

Simple kanban board by Jeff Lasovski
Demo Tecfa Board

(if time)

Discussion
  • Who uses project management tools in a project
  • Who uses project management tools for themselves

Quality assurance

Educational program development increasingly has to follow quality assurance (QA) principles.

QA typically defines:

  1. what quality education or quality programs are, e.g. QA defines and requires "learner centered education"
  2. that vision goals and learning goals must be formulated
  3. that program output must be measurable and measured
  4. that the institution must create structures and procedures for QA.
  5. that the institution must have an improvement policy (i.e. act upon evaluations)
Euniq assessment criteria

“Institutions should have a policy for quality assurance that is made public and forms part of their strategic management. Internal stakeholders should develop and implement this policy through appropriate structures and processes, while involving external stakeholders” (ENQA, 2018: 6 [13]

In Europe, most often European Standards and Guidelines for Quality Assurance in the Higher Education Area (ESG), 2015 is used to evaluate programs.

UNESCO Also produced internal and external Q/A guidelines, as well as world-wide and regional overview, search. See this Quality assurance in higher education overview, 2013. [14]

Pitfalls of Q/A:

  • Gaming the system
  • Looking at details and forgetting the picture
  • Stifling non-planned innovation and (useful) deviations
Discussion:
  • Who did encounter formal Quality Assessment (or Quality assurance) and what do you think of it
  • Could the implemented Q/A model be improved ?

Appreciative and flexible alternatives

Quality assurance can be somewhat appreciative. For example:

  • It can identify all positive elements (included non-planned ones)
  • It identifies problems as critical problems that should be addressed and provides some help to do so.

End of Monday lesson


Start of Wednesday lesson

Instructional program and course design principles

Educational Program Design

Educational program design, in principle, follows a series of steps that start with a vision and end with a program down to module design. For example, O'Neill (2015) [15] defines six steps:

  1. Vision / Educational Philosophy
  2. Curriculum Models
    • A shared view of teaching, learning and assessment approaches
    • Focus on product (general learning outcomes) or process (learning experience) models or both
  3. Program aims & outcomes
    • General program aims, i.e. a small general set of broad competencies
    • Specific program outcomes, i.e. knowledge, skills and attitudes that a typical student should acquire
  4. Program organisation and structure. There are different dimensions to look at, e.g.:
    • What is the scope of the program ?
    • Should course organization go from simple to complex, or from conceptual to applied ?
    • Integration of themes: How should various subjects be aligned ?
    • How is formative and summative assessment provided ?
  5. Teaching, Learning and assessment strategies
    • Intended outcomes, learning/teaching activities and assessment must be aligned ! (Biggs and Tang, 2007)
    • Respect basic principles of instruction and learning
  6. Module Design (see ADDIE for example)
    • Depending on the type of learning, teaching and learning strategies can be different but should align with general program aims.
Exploration and discussion of vision statements:
  • Identify a top 100 university, find the global vision statement
  • Identify a master program or a certified continuous education program and find its vision statement
  • Discussion according to what you found .....

Examples:

Instructional Systems Designs (ADDIE)

Instructional systems design is a method for designing courses and its units. It is popular for larger projects that include instructional designers. There are five components:

  • Analysis
  • Design
  • Development
  • Implementation
  • Evaluation

There exist two interpretations: (1) A waterfall model and (2) A non-linear model

A "traditional" waterfall model:

Waterfall model

A non-linear model:

Clark, D.R. (2004). Why Instructional System Design and ADDIE? Retrieved from http://www.nwlink.com/~donclark/hrd/sat1.html on nov 10 2016

Backwards Design

The model has 3 stages:

  1. Identify desired results (learning outcomes)
    • “What should students know, understand, and be able to do? What is worthy of understanding? What enduring understandings are desired?”
  2. Determine Acceptable Evidence (means to assess if learners have learnt)
    • “How will we know if students have achieved the desired results and met the standards? What will we accept as evidence of student understanding and proficiency?”
  3. Plan learning experiences and instruction. This includes:
    • definition of knowledge (know-that), skills and procedures (know-how) students ought to master
    • definition of materials
    • definition of learning /teaching activities (scenarios).

Optional Reading: https://files.ascd.org/staticfiles/ascd/pdf/siteASCD/publications/UbD_WhitePaper0312.pdf

There exist many other models. A simpler model is backwards design.

Discussion:
  • Who in the past did participate in a course design ?
  • Did you use an explicit design model/method ?

Program design check list

  • This list will allow participants to gradually define a project defining an educational program
  • Participants can make revisions to the list
  • At different times, participants will fill in some items of the checklist and make revisions during the whole length of the project.

Link: Educational program description grid (Draft v.4)

Moodle demo

  • Please connect to https://arbores.tech/lms/ (file with usernames and passwords is on UniGE Moodle)
  • Click on "My Courses"
  • Switch on editing (top right)

Reminder: Learn how to place most important URLs to a "Tool bar", available for all browsers but in different ways ....

Assignments

Week 1 mini project

Identify a real or imaginary educational program, e.g. a certificate like this one. A real project is preferred.

  • Create a copy of Educational program description grid (Draft v.4)
  • Fill in the first 2 sections (meta data, structural and organization variables) plus the first 4-5 items of section 3, pedagogical aspects of the checklist. As best as you can
  • You may build on top of prior work, i.e. continue working on module 2 (B.Class) projects
  • You are allowed submit group projects (max. 3 persons). One person will submit the project, the other(s) just mention in the Moodle assignment who submitted the project

Inspiration (if needed)

"Deadline" Sunday, March 20 2023, midnight

Submission:

  • Use the Moodle assigment to submit your work. Do NOT use the forum or email. One person will submit the project, the other(s) just mention in the Moodle assignment who submitted the project
  • Also post a link in the Moodle week 1 Forum so that fellow learners can see and comment your text.

Week 1 exchange

This is individual work, each student will be graded individually. Please use only the Moodle Forum for discussion. "Backchannels" cannot be graded.

  • Try to have an organized discussion by following the forum rules, e.g., choose meaningful names for threads, do not start new threads when another already exists.

1a) Discuss elements of the Educational program description grid in the Moodle UniGE Forum

  • Are there missing slots in the checklist ? If so, which ones ?
  • Do you suggest other modifications ?
  • Are there items that you do not understand ?

1b) Comment others' posts

2) Provide fellow learners with some feedback on their projects.

  • Make sur that all projects can be found, i.e. create one thread for each project (educational program description grid). You should post a link as soon as possible, otherwise you will not get any peer comments.
  • Each participant should at least make a comment for two other projects

I will grade quantity and quality of posts plus quantity and quality of exchange with others, i.e. you are expected to discuss others' post, not just posting your own opinions.

"Deadline" Wednesday, March 22 2023, midnight (slightly extended deadline)

Cited references

  1. Savoia, A. (2022). Pretotype It 10th Anniversary Edition - Pretotyping.org. Retrieved October 16, 2022, from https://www.pretotyping.org/pretotype-it-10th-anniversary-edition.html
  2. Preece Jenny, Yvonne Rogers and Helen Sharp (2002), Interaction Design, Wiley, ISBN: 0471492787
  3. Star, Susan Leigh, Griesemer, James R. (1989. Institutional Ecology, 'Translations' and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. Social Studies of Science, Vol. 19, No. 3, pp. 387-420
  4. Matt Cooper-Wright (2012). Prototyping and Boundary Objects https://medium.com/@matt_speaks/prototyping-and-boundary-objects-b469b63d5115
  5. Wenger, Etienne. (2000), Communities of Practice and Social Learning Systems, Organization, Volume 7(2): 225-246
  6. Mitchell R. Hammer, The Developmental paradigm for intercultural competence research, International Journal of Intercultural Relations, Volume 48, September 2015, Pages 12-13, ISSN 0147-1767, http://dx.doi.org/10.1016/j.ijintrel.2015.03.004. (http://www.sciencedirect.com/science/article/pii/S0147176715000279)
  7. K. Leung, S. Ang, M.L. Tan, Intercultural competence, Annual Review of Organizational Psychology and Organizational Behavior, 1 (2014), pp. 489–519 http://dx.doi.org/10.1146/annurev-orgpsych-031413-091229
  8. Cross, T.L., Bazron, B.J., Dennis, K.W. & Isaacs, M.R. (1989). Towards a culturally competent system of care. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center
  9. Cross, T. (2012). Cultural competence continuum. Journal of Child and Youth Care Work, 24, 83-85.
  10. McCowan, T., Omingo, M., Schendel, R., Adu-Yeboah, C., & Tabulawa, R. (2022). Enablers of pedagogical change within universities: Evidence from Kenya, Ghana and Botswana. International Journal of Educational Development, 90, 102558. https://doi.org/10.1016/J.IJEDUDEV.2022.102558
  11. Satir, Virginia, et. al., The Satir Model: Family Therapy and Beyond, ISBN 0831400781, Science and Behavior Books, 1991.
  12. Bates, A.W. (2000). Managing Technological Change: Strategies for College and University Leaders, San Franciso: Jossey-Bass.
  13. Huertas, E., & et al. (2018). Considerations for quality assurance of e-learning provision. https://www.enqa.eu/publications/considerations-for-qa-of-e-learning-provision/
  14. UNESCO (2013). Quality assurance in higher education, https://unesdoc.unesco.org/ark:/48223/pf0000222126.locale=en retrieved 10/2022
  15. O’Neill, G. (2015). Curriculum Design in Higher Education: Theory to Practice. In University College Dublin. Teaching and Learning (Vol. 9, Issue 2). University College Dublin. Teaching and Learning. https://researchrepository.ucd.ie/handle/10197/7137 or https://www.ucd.ie/teaching/t4media/curriculum_design_in_higher_education.pdf