2015 05-22 presentatie-calico_desmet & vandewaetere - def

Adaptive

language

learningChallenges & opportunities

Piet DESMET & Mieke VANDEWAETERE

CALICO 2015

ADAPTIVE

INSTRUCTION

CHALLENGES &

OPPORTUNITIES

WHAT

HOW

WHY

USE CASES

ADAPTIVE INSTRUCTIONI

WHAT IS ADAPTIVE INSTRUCTION?

“the method by which learners

are offered tailored instruction and support,

personalized to the individual

cognitive, affective and behavioral profile

of the learner.”

1

WHY ADAPTIVE INSTRUCTION?

Rationale

Learners differ

Cognitive

prior knowledge, metacognition, beliefs, goals, etc.

Affective

motivation, fear, anxiety, etc.

Behavioral

need for help and feedback, gaming the system, etc.

2

WHY ADAPTIVE INSTRUCTION?

Goal

To design supportive learning environments that account for individual

differences between learners (Shute & Zapata-Rivera, 2008)

To enhance performance & learning (Shute & Towle, 2003)

individualized instruction is superior to the one-size-fits-all approach

(Cohen, Kulik, & Kulik, 1982; Kadiyala & Crynes, 1998; Kulik, Kulik, & Bangert-Drowns, 1990)

2

HOW TO PROVIDE ADAPTIVE INSTRUCTION?3


Vandewaetere, Desmet & Clarebout 2011 / Vandewaetere & Clarebout, 2012


Learner

cognition

Learner

behavior

Learner

affect

Vandewaetere & Clarebout, 2012

ADAPT TO WHAT?Learner characteristics

Cognition

field (in)dependency, prior knowledge, learning style, information

skills, working memory capacity, etc

Affect

motivation, self-efficacy, frustration; relief, etc

Behavior

need for help, number of attempts, duration, etc.

3a


static

dual pathway

dynamic


Static: prior to/after interaction

Eg. Student’s starting level is defined by teacher or pretest

Eg. Update of student’s level after a series of completed

tasks/exercises – after logging out.

Dynamic: during interaction

Eg. Update of student’s level after every completed task/exercise

Eg. Update of student’s parameters after every interaction with system (hint use ->

update in learner model)

Dual pathway

Eg. Combination: starting level is defined by teacher (pre-defined student model) –

during interaction student model is updated

WHEN TO ADAPT?3b


sequence

form

content


Adaptive curriculum sequencing

- From computerized adaptive testing

- Based on IRT:

- a measurement theory where the probability of a correct answer depends

on person characteristics and characteristics of the items.

the difficulty of the items is adapted to the demonstrated level of knowledge

This presumes:

• Difficulty level of exercises is known

• Skills/knowledge level of student can be tracked and sketched reliably

• An item selection algorithm that offers the most suitable exercise (wrt to difficulty)

to the learner at a certain time in the learning process, taking account of the

learners’ knowledge level.

ADAPT WHAT?3c

Adaptive form & content representation

- Adaptive content presentation of learning objects

example: varying degrees of support

- with or without embedded support (eg. hints)

- with several degrees of feedback (eg. from 1|0 to faultspecific)

- with or without annotation of co-learners

- Adaptive form presentation of learning objects

example: multimodality adjusted according to context

- only text when slow connection

- no audio in noisy environment

- video enhanced with annotations of co-learners when fast connection and

much time

ADAPT WHAT?3c


learner-controlled

shared control

program-controlled


Program or system-controlled

System reasoner decides what content is offered.

Evaluation:

Lack of choice lowers motivation and fosters dependence (Hannafin & Rieber,

1989)

More risk to become dependent of pre-structuralized instruction (Elen, 2000)

Sparse interaction between learner & environment

High investment and development costs (eg. ITS)

ADAPT HOW?3d

Learner-controlled

Learner selects what content is offered.

Evaluation:

Not all learners can deal with choice: “the art of choosing”

Not for novice, low-motivated learners

Increases learners’ involvement, responsibility and self-regulation

strategies

ADAPT HOW?3d

Shared control

System preselects, learner chooses from preselection.

Best of both worlds

ADAPT HOW?3d

Device: Mobile vs desktop adaptive learning

Short duration – long duration

Short term – long term

Environment characteristics (eg. noise on train)

Certification or not

Quality of internet connection

Etc.

ADAPT WHEN? CONTEXT3e

ADAPTIVE

INSTRUCTION

CHALLENGES &

OPPORTUNITIES

WHAT

HOW

WHY

USE CASES

Adaptive item selection based on combination of judgment and

data (What?) [Wauters, Desmet, & Van Den Noortgate, 2012]

- IRT: estimation of item difficulty taking into account a learner’s ability.

Computationally intensive – a lot of data required

- Proportion correct

- ELO-rating system (Brinkhuis & Maris, 2010)

- Learners’ judgment “How difficult was the presented item to you?”

- One-to-many comparison by learners

- Expert ratings “How difficult do you think will this item be for your students”

This six techniques all provide reasonably accurate estimates of the difficulty of an

item, even with small sample sizes

Wauters K., Desmet P., Van Den Noortgate W. 2012. Item Difficulty Estimation: an Auspicious Collaboration Between Data and

Judgment. Computers and Education. Pergamon Press nr.58 , pp. 1183-1193 , ISSN 0360-1315

USE CASE 1 – ADAPTIVE ITEM SEQUENCINGII

Illusion of adaptivity might be as effective as adaptivity

(to what?) [Vandewaetere, Clarebout, Desmet, 2011]

Adaptive instruction is motivating

Illusion of adaptive instruction is also motivating

Learners’ perceptions are important in the relation adaptive

instruction – motivation.

USE CASE 2 – ADAPTIVITY & MOTIVATIONII

adaptive

instruction

↗ learning

outcomes

perception

beliefs

motivation

Vandewaetere, M., Desmet, P., Clarebout, G. (2011). The contribution of learner characteristics in the development of

computer-based adaptive learning environments. Computers in Human Behavior, 27, 118-130.

Learner control as a means to provide adaptive instruction

(how?) [Vandewaetere & Clarebout, 2011]

LC has to be perceived by learners:

additional instruction of LC strengthens the perception

of control

higher perception of control is related to higher learning

outcomes and motivation

Study in language learning – N=165, age 18-20

English tenses – 3 conditions: NC, LC, LC with additional

instruction of control

USE CASE 3: ADAPTIVITY & LEARNER CONTROLII

Vandewaetere, M., Clarebout, G. (2011). Can instruction as such affect learning? The case of learner control.

Computers and Education, 57(4), 2322-2332.


[Vandewaetere, Clarebout, Desmet, 2011]

USE CASE 3II


USE CASE 3II

Instruction of Learner control as a means to provide adaptive

instruction

Direct effect of instruction of LC on perceptions, which in

turn were related to motivation

Additional instruction of control: higher satisfaction with

control, higher interest/enjoyment, higher perceived

competence and higher interest in learning.

USE CASE 3II

ADAPTIVE

INSTRUCTION

CHALLENGES &

OPPORTUNITIES

WHAT

HOW

WHY

USE CASES

Adaptive feedback/support (adaptive representation) &LC

(What? & How?))• Shed light on the effect of giving learners control on feedback levels. Different behavior

and use of feedback for learners having low and high prior knowledge, and for learners

with low and high self-regulated learning skills and motivation. Also, we expect the

selection of feedback to be different when learners ask feedback on difficult versus easy

items.

• Hypothesis 1.1: There is a main effect of learners’ prior knowledge on the type of requested feedback. Learners with

low prior knowledge will ask more frequently for detailed feedback as compared to learners with higher prior

knowledge.

• Hypothesis 1.2: There is a main effect of item difficulty level on the type of requested feedback. After completing a

difficult item, learners will ask more frequently for detailed feedback as compared to completing an easy item.

• Hypothesis 1.3: There is an interaction effect between item difficulty and prior knowledge. Learners with higher prior

knowledge will request less times detailed feedback after completing a difficult item than learners having lower prior

knowledge.

• four types of feedback:• Faultspecific (wrong/correct + if wrong: correct answer + specific error feedback)

• General (wrong/correct + if wrong: correct answer + general attention remark)

• Correct answer (wrong/correct + if wrong: correct answer)

• Binary (wrong/correct)

OPPORTUNITIES: SOME EXAMPLESIII

Adaptive media enhancement (adaptive representation)

(What?)

OPPORTUNITIESIII

Different types of enhanced audio-visual input may serve different learning goals (e.g.,

improve comprehension, stimulate learning of formulaic language). Yet, the visualisation can also

be adapted to learners’ profile.

• H 1: Different types of enhancement are chosen in function of the learner profile (proficiency level,

motivation, etc.).

• H 2: Different types of enhancement can be situated on an implicational scale going from less complex to

more complex viewing experiences.

If the aforementioned hypotheses are confirmed, adaptive viewing paths (such as the one

suggested above) can be established in function of different learner profiles and objectives.

L1 subtitles

L2 subtitleswith L1 gloss

L2 subtitles

L2 keywords

CHALLENGESIII

Big data, big opportunities

From manually entering data to online massive storage

From self-reporting data to behavioral data

From single measurements to longitudinal measurements

From inaccessible to everywhere

Research domains:

-Learning analytics

-Educational data mining

Hype cycle for education (Gartner, 2013)

Contact

Piet Desmet

[email protected]

http://wwwling.arts.kuleuven.ac.be/franling_e/pdesmet

be.linkedin.com/in/pietdesmet

@PietDesmet

ITEC

www.kuleuven-kulak.be/itec