Program design and implementation (CAS HEiE) - week 2

Week 2 - Elements of educational program design

• Interactive Class: Monday October 24: 17h00 - 19h15
• Interactive Class: Wednesday October 26: 17h00 - 19h15
• Public tutoring session: Friday October 28: 17h-19h

Short week 1 discussion

Short discussion of the program design checklist:

• Sections (Meta should be just editing information, e.g. like a library record)
• Discussion
• Organization of work (do not present to the teacher, but engage with others)
• .....

We also will discuss evaluation of student participation (later today) when we discuss evaluation rubrics

Needs analysis, competence framework and learning goal definition

• Needs analysis should result in some systematic document that describe what participants in a program should learn. It usually describes what a typical learner upon successful completion are able to do, i.e. competencies that are broad definitions of knowledge, skills (know how) and attitudes.
• In order to plan the program, the document also should describe intermediary competency levels.
• If competencies are defined in an operational way, they describe learning outcomes (learning goals)
• Learning management systems, such as Moodle allow defining competence frameworks (also called competence maps) and learning plans that map courses and activities with competences.

Introduction

What is in a competence ?

• Usually, a set of knowledge and know-how/skills, sometimes attitudes, that allow solving a specific class of problems.

A spiral view of competence development (DKS)

• Usually, a competence has dimensions. In that case, the term competence framework can be used.

How can competences be structured

There are widely different ways to describe a set of comptencies.

Stoof et al (2007:349) acknowledge that competence can be defined in manners that suit specific situations. They however point out six important dimensions found in an analysis of 16 competence maps and that designers should think about when creating competence maps:

1. levels: can competencies can be subdivided, e.g. in starting level, advanced level, and experienced level ?
2. context: are competencies connected to real world phenomena, e.g. example tasks, roles, functions or situations ?
3. relationships, are competencies are related to each other ?
4. elements, are competencies composed of several parts, e.g. knowledge, skills and attitudes ?
5. output, can competencies lead to specific outcomes, such as a product or service, or behavior in general ?
6. kinds, are there more competencies than just professional competencies, such as learning competencies, career competencies and competencies that are general to all kinds of professions ?

Moodle has yet another list. It allows creating a nested list from:

Example of a simple table to create a competence framework

• Competencies are usually defined in terms of levels (at least three, otherwise you only will have "pass"/"fail")
• Competence levels are defined for each "topic" (dimension) that is part of the program. These can be well defined subject areas or transversal skills.
• Each topic is further divided into sub-topics (sub-dimensions)

In the following summary table, dimensions (also called "areas", "topics", "subject matters" are on one axis, levels on another. The cells then can be filled with description of the competence and other useful information, e.g. what should be observed, behaviorist sentences.

Simple, abstract competency framework

Level of competency	Dimension 1			Dimension 2		Dimension 3		Dimension ...
	sub-dimension 1.1	sub dimension 1.2	sub dimension 1.3	1	2	3.1	3.2	x.y	x.y
Level 1
Level 2
Level 3

In order to use such a table in a system like Moodle, you will have to create a list. There are two basic options:

(1) Order according to "dimensions". Examples of a few items for "Digital design and fabrication"-

• Digital Design and Fabrication
  • Use of machines
    • 3D printer
      • Level 1: Use PLA with standard slicer settings
      • Level 2: Use other filament and select appropriate settings
      • Level 3:
    • Laser cutter
      • ......
    • Embroidery machine
      • ......
  • Use of design programs
    • 2D drawing
      • ........
    • 3D modeling
• Creativity
  • Create designs
  • Find "making" solutions to known problems

(2) Order according to "level"

• Novice
  • 3D printing
    • Can print simple objects using standard settings
    • Clean print plate and remove prints
  • Laser cutter
    • Can send 2D drawings to a laser cutter selecting a profile from preselects
• Experienced
  • 3D printing
    • Can work with PLA, PETG, etc and select appropriate settings
    • Changes filament

• Expert

Competence maps

Competence frameworks are sometimes called competence maps since taxonomies like the tables above and below define at least implicitly some relationships. A competence map or competency map defines a tree or network of connected competences that are in turn operationalized in terms of learning objectives. In simpler terms, it describes what a learner will know and be able to do-

According to IMS Reusable Definition of Competency or Educational Objective - Best Practice and Implementation Guide , an elearning standards organisation, maps are how skills and competencies or competency definitions can be aggregated to form more comprehensive skills and competencies, or decomposed into component skills or competencies. Taxonomies are simple maps in the form of trees. Also "Competencies must be related programs, to courses and modules, assessments, course materials, and learners' records of achievement" (Introduction to the 1EdTech CASE Standard.)

Example: Common European Framework of Reference (CEFR) for language skills

The CEFR divides learners into three broad divisions that can each be further divided into two levels; for each level, it describes what a learner is supposed to be able to do in reading, listening, speaking and writing. The following table indicates these levels. Dimensions (listening, reading, speaking and writing) are not very clearly organized in this table.

Level group	Level	Description
A Basic user	A1 Breakthrough	Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce themselves and others and can ask and answer questions about personal details such as where they live, people they know and things they have. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.
	A2 Waystage	Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of their background, immediate environment and matters in areas of immediate need.
B Independent user	B1 Threshold	Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise while travelling in an area where the language is spoken. Can produce simple connected text on topics that are familiar or of personal interest. Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans.
	B2 Vantage	Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in their field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.
C Proficient user	C1 Advanced	Can understand a wide range of demanding, longer clauses and recognise implicit meaning. Can express ideas fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.
	C2 Mastery	Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express themselves spontaneously, very fluently and precisely, differentiating finer shades of meaning even in the most complex situations.

More information:

• (A simple) Common European Framework of Reference (CEFR) self-assessment grid

Example: Europass certificate supplement: Competence map for applied social studies, in Ireland

The following list divides what is learned in knowledge, know-how/skills and competence

The purpose of this award is to enable the learner to acquire the knowledge, skills and competence to work under supervision in range of social care settings or to progress to further and or higher education and training. The holder of the certificate is able to:

knowledge

• demonstrate a broad range of knowledge related to the social, psychological and cultural issues within the field of applied social studies;
• demonstrate knowledge of some theoretical concepts and abstract thinking pertinent to social studies with significant depth in certain areas;

know-how and skills

• demonstrate a broad range of practice, interpersonal and communication skills in providing for the holistic needs of individuals in the social care environment;
• exercise judgment in selecting appropriate procedural responses to routine, varied and unfamiliar situations and challenges within a social care setting;

competence

• demonstrate knowledge and skills within a range of varied and specific social care contexts and settings.;
• contribute to the assessment, planning, implementation and evaluation of a range of interventions in support of the holistic needs of the individual;
• take responsibility for own learning in a structured context;
• reflect on personal practice to inform self-understanding and personal development.

What do you think about the distinction between knowledge, know-how and skills, and competence ?

Example: DigiCompEdu competency framework

DigiCompEdu is a fairly detailed framework to describe ICT in education skills for teachers. “The teaching professions face rapidly changing demands, which require a new, broader and more sophisticated set of competences than before. [...] DigiCompEdu is a scientifically sound framework describing what it means for educators to be digitally competent. It provides a general reference frame to support the development of educator-specific digital competences in Europe. DigCompEdu is directed towards educators at all levels of education, from early childhood to higher and adult education, including general and vocational education and training, special needs education, and non-formal learning contexts.” (Digital Competence Framework for Educators (DigCompEdu), retr. oct 2022)

It has 138 items and is organized according to six levels like CEFR language skills but also in terms of 22 "areas".

The six levels summarized

• Newcomer (A1) : aware of the potential of digital technologies for enhancing pedagogical and professional practice.
• Explorer (A2): ware of the potential of digital technologies and are interested in exploring them to enhance pedagogical and professional practice.
• Integrator (B1): experiment with digital technologies in a variety of contexts and for a range of purposes, integrating them into many of their practices.
• Expert (B2): use a range of digital technologies confidently, creatively and critically to enhance their professional activities.
• Leader (C1): have a consistent and comprehensive approach to using digital technologies to enhance pedagogic and professional practices.
• Pioneer (C2): question the adequacy of contemporary digital and pedagogical practices, of which they themselves are Leaders.

The 22 areas are:

source: DigCompEdu framework

For more details see:

• Digital Competence Framework for Educators (DigCompEdu)
• Proficiency levels
• DigCompEdu CheckIn for Higher Eucation (self test, EU registration is required)

A Moodle implementation:

• https://arbores.tech/lms/admin/tool/lp/competencies.php?competencyframeworkid=6&pagecontextid=1
  (access restricted link)

Discussion: Is it possible to both an Expert or Leader and a Pioneer ?

Some discussion

Some competency frameworks may have issues that may need to be addressed:

• Elements may be missing
• "Whole" competences, e.g. carry out given projects may include extra skills that are not present
• Levels may not always be the appropriate ones
• Developmental potential is not measured
• Use of a competence is not measured in some grids
• Attitudes are missing.

The following rather shows a developmental framework for digital learners, based on a "professional identity" concept as defined by Wenger.

Modified Beetham and Sharpe (Source: Developing digital literacies (JISC, 2014)‘pyramid model’ of digital literacy development model)

First look at Moodle

The instructor will demonstrate:

• Basic course creation functionality
• Competence frameworks
• Assignments and evaluation rubrics.
  • How to evaluate forum participation ?

Moodle resources:

• Teacher quick guide
• Managing a Moodle course
• Assignment activity
• Grades, in particular Advanced grading method types

Create a grading rubric

Grading rubrics allow grading with explicit grading criteria and performance level. Advantages:

• Better consistency
• Explicit criteria
• Gain of time
• Formative function (in particular if the rubric is communicated beforehand and/or discussed)

Ideally, rubrics should describe what is observable, e.g. what the learner does and not what he/she does not do.

	Performance measure	Performance measure
Criterion 1	Performance indicator	Performance indicator
Criterion 2	Performance indicator	Performance indicator
Criterion 2	Performance indicator	Performance indicator

Ideas for rubrics

• Rubystar is a free tool to help teachers create and share quality rubrics.

To create grading rubrics in Moodle:

1. Create an assignment

2. Open the "Grade" section and define grading with points or a continuous scale and select "rubric"

Create an assignment with a grading rubric

3. Save and display

4. Create or edit the rubric.

To edit the rubric again,

Re-edit the grading rubric

Student tracking and followup

Moodle includes several tools for student tracking, see:

• Tracking progress

In this class we will mainly look at competencies (not necessarily to be used in Moodle later) and badges. As a future user of Moodle, you also should understand the simple course and activity completion concepts.

Create and use competency frameworks

In a "normal" Moodle, teachers cannot create competency frameworks. They can only tie achievements to competency items. Creating and managing competency frameworks and learning plans is very complicated and no so well documented.

Through site administration -> Competencies, a user with permissions can:

• Create and import competency frameworks
• Create learning plans, i.e. collections of items from various compettencies that will define what an individual learner or all learners in a given cohort should learn

In the Arbores Tech Moodle I did grant permission to create and use competences:

• Please make sure to only edit your competence framework
• Before entering it in Moodle, do create a paper version, since the interface is not obvious to use.

To access the competencies management tool, select under more then Manage competencies and frameworks.

Create and manage a competence framework

Moodle resources:

• Competency frameworks Overview page at Moodle
• Learning Plans Overview page.
• Competencies FAQ

Moodle resources for Moodle administrators (in case you have your own Moodle !)

• Managing roles

Program structure

(TBA)

Program coordination and learner support

Coordination

A program needs to be coordinated.

Tutoring structure

Good distance teaching programs do have a tutoring structure.

Functions of a tutor:

Organization

• At least one regular meeting
• Tutoring on demand

Discussion: Can tutoring be done without tutors ?

Global followup

• Global learner support (coaching, mentoring, advising, follow-up)

Community building

• Community building (-> Do profiles in Moodle, whatsapp, etc.)

Program evaluation

Evaluation is discussed in the last module of this CAS. However, good design philosophy (e.g. quality assurance) does require that evaluation is part of the design. Since it is a design factor or at least influences a design, you should at least think about it.

• Program and course evaluation (see also last module): Zaharias, P. (2004).
• Evaluation criteria, e.g. https://unesdoc.unesco.org/ark:/48223/pf0000382843
• Learning evaluation philosophy, policy and method

Certification

(TBA)

• accredited vs. micro-acreditations https://openbadges.org/

Using Badges

(TBA)

Assignments

Week 2 mini project

• Define a simple competency framework
• Map competencies to a program, i.e., a list of some courses
• Define learner support framework
• Suggest certification / collaboration with other institutions if appropriate.

Week 12 exchange

• Comment on the competency framework of two or more other participants. Make sure that everyone gets at least one feedback.
• Engage in discussion