mmoonniittoorriinngg && eevvaalluuaattiioonn …sals.gov.za/docs/pubs/report.pdf ·...
TRANSCRIPT
MMOONNIITTOORRIINNGG
&&
EEVVAALLUUAATTIIOONN RREEPPOORRTT
FFaacciilliittaattoorr aanndd
PPaarrttiicciippaanntt FFeeeeddbbaacckk::
CCaappaacciittyy BBuuiillddiinngg
PPrrooggrraammmmee ffoorr
MMeemmbbeerrss ooff PPaarrlliiaammeenntt
aanndd PPrroovviinncciiaall
LLeeggiissllaattuurreess
CCoommppiilleedd bbyy
MMoonniittoorriinngg && EEvvaalluuaattiioonn CChhiieeff DDiirreeccttoorraattee
MMaayy 22001111
TABLE OF CONTENTS
EXECUTIVE SUMMARY .................................................................................................................. I
ACKNOWLEDGEMENTS .............................................................................................................. III
ACRONYMS ................................................................................................................................... IV
LIST OF TABLES............................................................................................................................ V
CHAPTER 1 .................................................................................................................................... 1
BACKGROUND AND INTRODUCTION TO THE EVALUATION .................................................. 1
1.1 BACKGROUND AND INTRODUCTION........................................................................................................ 1 1.2 AIM OF THE EVALUATION ...................................................................................................................... 4 1.3 LIMITATION OF THE EVALUATION ........................................................................................................... 5 1.4 LAYOUT OF THE REPORT ...................................................................................................................... 5
CHAPTER 2 .................................................................................................................................... 6
DESIGN AND METHODOLOGY .................................................................................................... 6
2.1 METHODOLOGY ................................................................................................................................... 6 2.2 INSTRUMENT DESIGN ............................................................................................................................ 6 2.3 SAMPLING STRATEGY .......................................................................................................................... 7 2.4 DATA CAPTURING AND ANALYSIS .......................................................................................................... 7
CHAPTER 3 .................................................................................................................................... 9
FINDINGS OF THE EVALUATION ................................................................................................. 9
3.1 PARTICIPANTS’ SURVEY: REACTION EVALUATION QUESTIONNAIRE (REQ) ...................................... 9 3.1.1 COURSE CONTENT ...................................................................................................................... 9 3.1.1.1 QUANTITATIVE RESULTS .............................................................................................................. 9 3.1.1.2 QUALITATIVE RESULTS ...............................................................................................................11 3.1.2 THE FACILITATOR.......................................................................................................................12 3.1.2.1 QUANTITATIVE RESULTS .............................................................................................................12 3.1.2.2 QUALITATIVE RESULTS ...............................................................................................................15 3.1.3 LOGISTICS, VENUE AND FOOD ......................................................................................................16 3.1.3.1 QUANTITATIVE RESULTS .............................................................................................................16 3.1.3.2 QUALITATIVE RESULTS ...............................................................................................................18 3.1.4 OVERALL RESULTS ....................................................................................................................20 3.1.4.1 QUANTITATIVE RESULTS .............................................................................................................20 3.1.4.2 QUALITATIVE RESULTS ...............................................................................................................22 3.2 FACILITATOR SURVEY: FACILITATORS FEEDBACK FORM (FFF) .....................................................24 3.2.1 COURSE CONTENT .....................................................................................................................24 3.2.1.1 QUANTITATIVE RESULTS .............................................................................................................24 3.2.1.2 QUALITATIVE RESULTS ...............................................................................................................27 3.2.2 FACILITATOR ASSESSMENT OF PARTICIPANTS ..............................................................................28 3.2.2.1 QUANTITATIVE RESULTS .............................................................................................................28 3.2.2.2 QUALITATIVE RESULTS ...............................................................................................................33 3.2.3 FACILITATION .............................................................................................................................34 3.2.4 LOGISTICS .................................................................................................................................35 3.2.4.1 QUANTITATIVE RESULTS .............................................................................................................35 3.2.4.2 QUALITATIVE RESULTS ...............................................................................................................39 3.2.5 FACILITATORS’ SUGGESTIONS REGARDING THE IMPROVEMENT OF THE CAPACITY BUILDING
PROGRAMME FOR MEMBERS OF PARLIAMENT AND PROVINCIAL LEGISLATURES ...............................39
CHAPTER 4 .................................................................................................................................. 41
RECOMMENDATIONS AND CONCLUSION ............................................................................... 41
4.1 INTRODUCTION .............................................................................................................................41 4.2 RECOMMENDATIONS ...................................................................................................................41 4.3 CONCLUSION ................................................................................................................................42
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
I
EXECUTIVE SUMMARY
The Capacity Building Programme for Members of Parliament and Provincial Legislatures is the
initiative of the South African Legislative Sector under the leadership of the Speakers’ Forum.
The Speakers’ Forum resolved to work with the Public Administration Leadership and
Management Academy to develop and facilitate the development of a standardised capacity
programme relevant to the core responsibilities of members.
The programme comprises the following five modules:
Information Communication Technology Tools
Action research for the legislative process
Media Communication
Democratic governance and the legislative process
Values based leadership and decision making
The instruments used for this evaluation were the Facilitator Feedback Form which was
completed by each facilitator after every training intervention and the Reaction Evaluation
Questionnaire which was completed by every participant on completion of every training
intervention.
This evaluation report covers twenty five training interventions and four hundred and thirty five
Reaction Evaluation Questionnaires which were received at PALAMA.
The quantitative evaluation findings are presented per evaluation instrument and per variable
evaluated in the table below:
Evaluation Instrument Variable Evaluation Finding
Reaction Evaluation Questionnaire
Course content High participant satisfaction levels.
Facilitator High participant satisfaction levels.
Logistics High participant satisfaction levels.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
II
Evaluation Instrument Variable Evaluation Finding
Overall Result High participant satisfaction levels.
Facilitator Feedback Form
Course content High positive facilitator ratings, although certain issues pertaining to the content were raised.
Participants Participants were mostly at the appropriate level for the specific module. The number and mix of participants were mostly appropriate.
Facilitation The facilitators mentioned specific issues which they would like to improve on as well as elements which they would do differently during their next presentation.
Logistics High positive facilitator ratings.
In the quantitative feedback the participants and facilitators made specific recommendations
regarding programme improvements.
The evaluation report is concluded with recommendations based on the summative evaluation
findings.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
III
ACKNOWLEDGEMENTS
Participants
The participants, who completed the Reaction Evaluation Questionnaires after each of the
modules of the Capacity Building Programme for Members of Parliament and Provincial
Legislatures and provided PALAMA with valuable information to improve the quality of their
programme.
Facilitators
The facilitators, for their comments and recommendations as well as presenting a
professional image of the Academy to clients and stakeholders.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
IV
ACRONYMS
FFFs Facilitator Feedback Forms
FPE Report Facilitator and Participant Evaluation Report
ICT Information Communication Technology
M&E Monitoring and Evaluation
MPs Members of Parliament
PALAMA Public Administration Leadership and Management Academy
REQs Reaction Evaluation Questionnaires
SALS South African Legislative Sector
TMS Training Management System
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
V
LIST OF TABLES
PAGE
Table 2.1 Number of participants who returned their REQs per module 7
Table 3.1 Participant perceptions regarding the course content 10
Table 3.2 Participant comments regarding the course content 11
Table 3.3 Participant perceptions regarding the facilitation 13
Table 3.4 Participant perceptions regarding the facilitators’ Public Service
knowledge
14
Table 3.5 Participant perceptions regarding the facilitation of the course 15
Table 3.6 Participant perceptions regarding the logistics, venue and food 17
Table 3.7 Participant comments regarding the logistics, venue and food 18
Table 3.8 Overall perceptions of Participants 20
Table 3.9 Participant comments regarding the overall rating of the course 22
Table 3.10 Facilitator perceptions regarding the length of the course 25
Table 3.11 Facilitator perceptions regarding pacing of the course materials 25
Table 3.12 Facilitator perceptions regarding the relevance of the materials to the
Public Sector
26
Table 3.13 Facilitator comments regarding the course content 27
Table 3.14 Facilitator perceptions regarding the appropriate level of the participants 29
Table 3.15 Facilitator perceptions regarding the number of participants who
attended
30
Table 3.16 Facilitator perceptions regarding the mix of participants 31
Table 3.17 Facilitator perceptions regarding the attendance of the participants 32
Table 3.18 Facilitator comments regarding the attendance of the participants 33
Table 3.19 Facilitator comments regarding the facilitation 34
Table 3.20 Facilitator perceptions regarding the organisation of the course 35
Table 3.21 Facilitator perceptions regarding the suitability of the training venue 37
Table 3.22 Facilitator perceptions regarding the facilities, food and accommodation 38
Table 3.23 Facilitator comments regarding the Logistics 39
Table 3.24 Facilitator suggestions regarding the improvement of the Capacity
Building Programme for Members of Parliament and Provincial
Legislatures
39
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
1
CHAPTER 1
BACKGROUND AND INTRODUCTION TO THE EVALUATION
1.1 Background and Introduction
The Capacity Building Programme for Members of Parliament and Provincial Legislatures is the
initiative of the South African Legislative Sector under the leadership of the Speakers’ Forum.
The Speakers’ Forum of South Africa is a voluntary association which composed of the Speaker
and Deputy Speaker of the National Assembly, Chairperson and the Deputy Chairperson of the
NCOP and Speakers and Deputy Speakers of all nine Provincial Legislatures. The Speaker of
the National Assembly is the chairperson of the Speakers’ Forum and acts as the contracting
authority on behalf of the South African Legislative Sector (SALS).
One of the Speakers’ Forum’s key objectives is to ensure that Members of Parliament and
Provincial Legislatures are fully capacitated and enabled to exercise their role and
responsibilities. A programme is being developed to ensure that all members are exposed to a
standardised and accredited capacity building programme.
The Members’ Capacity Building programme is a unique professional training and development
programme sponsored by the Speakers’ Forum. It is a programme that will build on the lessons
and gains made on the previous sector programmes.
New and innovative ideas were received from Members during the training needs analysis
exercise conducted in Parliament and Provincial Legislatures in September 2009 by the Public
Administration Leadership and Management Academy (PALAMA) Research Chief Director, Dr
Andre Kraak.
The Speakers’ Forum resolved to work with the Public Administration Leadership and
Management Academy to develop and facilitate the development of a standardised capacity
programme which would be relevant to the core responsibilities of members.
The learning outcomes and programme structure of the Capacity Building Programme for
Members of Parliament and Provincial Legislatures are described as follows in the programme
learner guide:
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
2
“Learning Outcomes
The overall purpose of the Introductory Programme is to enhance the professional legislative and oversight capabilities of members of legislatures and parliament and provide access to the Postgraduate Certificate Programme. On completion of the programme, participants will be able to:
Lead more effectively in holding government to account.
Analyse policy, legislation, strategy and implementation.
Make judgments and interventions.
Apply technology in effective communication and decision making.
Programme Structure
The Introductory Programme will utilise an action learning case based approach in order to ensure the learning process supports the work of members. The following key learning principles apply:
The learning, application and reflection cycle will be utilised to ensure that participants have sufficient time to practice new skills and test new ideas and concepts.
Professional skills will be reinforced through the use of assignments structured to support learning in the programme. For example, for media communication, participants will be required to engage with the media in a simulated setting.
A variety of learning methods will be utilised on the programme including storytelling, workshops, lecturing, simulations and case based discussion.
The modules will take place in block with time in between for application.
Participants will be assessed through an applied assignment in order to demonstrate their competence.
The Introductory Programme comprises 5 modules which are described below. Each module is described with the key learning outcomes and content. Information Communication Technology Tools
This module will provide a basic introduction to the Microsoft Office, the power of computing technology and virtual office management.
Learning outcomes:
Participants will be able to:
Understand how computer technology and software packages assist in their work.
Use outlook and the internet to assist their work.
Identify software packages where they require further development.
Content: Basic computer skills (introduction to packages and what they can do).
Using cell phones to support their work and manage their time.
Introduction to the computer and its various applications and programmes.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
3
Action research for the legislative process
This module provides the essential research tools which will enable greater engagement in policy making and oversight.
Learning outcomes:
Participants will be able to:
Identify and articulate problems and issues.
Collect and analyse information for the legislative process.
Use analysis to develop convincing arguments.
Content: Using enquiry in day to day work.
Data gathering methods (rapid appraisal, appreciative inquiry, technology, libraries, interviews, public participation), reading, assessing (credibility and validity) and collating information.
Analytical skills and constructing arguments.
Understanding and using research ethically and effectively.
Media Communication
This module will enable members to use written and verbal communication to get their message across and prepare them to engage with the media more effectively.
Learning outcomes:
Participants will be able to:
Effectively communicate and defend their position to their committees, the legislatures, constituents and the media.
Content: Understanding and handling the media.
Communication as an art including speech writing and public speaking: Understanding the audience
Getting the message right
Text/discourse analysis
Dealing with questions
Relationships management
Communications strategy Democratic governance and the legislative process
This module would build off previous induction programmes and provide a foundation and context for professional work of members.
Learning outcomes:
Participants will be able to:
Understand the context in which the legislative process takes place.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
4
Understand the legislative process (from the perspective of the legal and legislative regulatory framework).
Content: Understanding the challenges and imperatives of the democratic developmental state and developmental governance (governance context, political economy).
The Constitution and separation of powers.
Role of legislatures in contributing to good governance.
Process of law making and public policy making.
Introduction to the budgeting process and parliament's oversight function.
Legislative review process.
Values based leadership and decision making
This module provides the space for members to reflect on their leadership approach and commitments and their implications for decision making.
Learning outcomes:
Participants will be able to:
Manage time and work more effectively.
Understand themselves, their leadership style and their impact on others.
Content: Emotional intelligence and self-knowledge.
Work life balance - managing multiple roles as members, managing constituency offices and family responsibility.
Scope of leadership.
Building a personal brand (trust, attitudes).
Dealing with power, criticism and complexes (ethics, leadership authority, credibility and integrity).
Leadership development planning.”
1.2 Aim of the evaluation
The aim of the evaluation of the Introductory Capacity Building Programme for Members of
Parliament and Provincial Legislatures is to provide feedback to the stakeholders regarding the
quality of the training provided. The training programme was evaluated on the following basis:
The quality of the programme content.
The feedback received from the facilitators.
The feedback received from the participants.
The appropriateness of the logistical arrangements for the training sessions held.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
5
1.3 Limitation of the evaluation
The evaluation report is based on the surveys completed by the participants, i.e. the Reaction
Evaluation Questionnaires (REQs) and the Facilitator Feedback Forms (FFF) which were
completed after every training session. The data was analysed, using both quantitative
(descriptive statistics) and qualitative (thematic analyses) analytical techniques.
No interviews were conducted with any of the participants and facilitators.
1.4 Layout of the report
The evaluation report is divided into chapters which gives the reader a logical format of the
results, recommendations and conclusion that emerged from the survey. Each chapter is listed
with a brief synopsis of its focus below:
Chapter 1 focuses on the introduction and background, the aim and limitation of the
evaluation.
Chapter 2 examines the design and methodology of the evaluation, i.e. the design
process of the instruments used to collect the evaluation data, the sampling strategy, the
data capturing and the measures that were implemented to ensure the integrity of the
data being captured.
Chapter 3 focuses on the results of the evaluation. It is structured to allow for separate
reporting of qualitative and quantitative data analysed of participants and facilitators.
Chapter 4 outlines the conclusion and describes the recommendations on the variables
evaluated.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
6
CHAPTER 2
DESIGN AND METHODOLOGY
2.1 Methodology
As no interviews were conducted, the evaluation methodology is based on the surveys that
were completed by the participants, i.e. the Reaction Evaluation Questionnaires (REQs) and the
Facilitator Feedback Forms (FFF). These surveys were completed after every training session.
2.2 Instrument design
The Chief Directorate: Monitoring and Evaluation at PALAMA developed the instruments that
were utilised for the data collection of the training interventions which took place as part of the
Introductory Capacity Building Programme for Members of Parliament and Provincial
Legislatures.
The PALAMA Reaction Evaluation Questionnaire (REQ) formed the basis of the participant
analysis that was conducted. The variables evaluated and reported on from the REQs are listed
below:
the course content;
the facilitator;
the logistics; and
the overall result.
The PALAMA Facilitator Feedback Form (FFF) formed the basis of the facilitator analysis
that was conducted. The variables evaluated and reported on from the FFF are listed below:
the course content;
the participants;
the facilitation; and
the logistics.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
7
2.3 Sampling strategy
To give effect to the stated aim of the evaluation, the following sampling strategy was
implemented:
All 435 participants who attended five modules of the Introductory Capacity Building
Programme for Members of Parliament and Provincial Legislatures and completed the
Reaction Evaluation Questionnaires (REQs) were included in the evaluation. A
breakdown per module in Table 2.1 indicates the number of participants who returned
their REQs as follows:
Table 2.1 Number of participants who returned their REQs per module
Module Presented Number of
training interventions
Number of REQs received
Information Communication Technology (ICT) Tools
3 51
Democratic governance and legislative process
4 77
Media Communication 6 89
Value based leadership for decision making 5 88
Action Research for the legislature process 7 130
Total 25 435
Eleven of the 12 facilitators involved in the training interventions completed a Facilitator
Feedback Form (FFF) and were included in the evaluation. Certain facilitators facilitated
more than one training intervention. Facilitator Feedback Forms were received for 23 of
the 25 training interventions which were presented.
2.4 Data capturing and analysis
A four point Likert Scale (Excellent, Good, Average and Poor) was used in the REQs and the
Facilitator Feedback Forms to capture the ratings of the participants and facilitators regarding
the variables that were evaluated.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
8
In order to ensure that the highest levels of scientific rigour were associated with the
administration of these instruments, facilitators were briefed during the Training of Trainers
(ToT) sessions in respect of the correct administering processes and procedures of the
monitoring and evaluation instruments.
In order to ensure the integrity of the data and analysis, the process of data capturing was
subject to a process of rigorous quality control for quality assurance purposes. A brief outline of
the procedures followed is listed below:
The raw data (from the REQs and FFFs) was captured on the PALAMA Training
Management System.
The data on the TMS was captured in binary format to ensure simplicity and accuracy of the
capturing process – the system allowed only one data input per field, and this input had to
be in the binary format.
The captured data was summarised per training intervention by the TMS and was exported
to a Facilitator and Participant Evaluation (FPE) Report after which this summarised data
per training intervention was entered into a spread sheet and quality control was effected
on 30% of all data captured, that is:
The accuracy with which the data had been captured was compared to the original
responses on the hard copy of the FPE report.
Data cleaning was conducted to ensure that all the descriptive statistics add up to 100%.
The integrity of all “formula driven back-end processing” on the captured data was
checked manually.
Feedback on data integrity was provided to the data capturers during scheduled weekly
meetings. During these meetings minutes were taken to record inputs and feedback on
issues that affected the data integrity.
The summarised quantitative data per training intervention collected from the TMS was
captured on a spread sheet and basic descriptive statistical analytical techniques, namely the
calculation of frequencies and percentages were conducted. The results of the quantitative data
analysis were summarised in tabular format to highlight the key trends.
The qualitative data collected was captured electronically on a spread sheet and a thematic
analysis of the content was conducted. The results of the qualitative data analysis, both positive
and negative comments were summarised and represented in tables.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
9
CHAPTER 3
FINDINGS OF THE EVALUATION
3.1 Participants’ Survey: Reaction Evaluation Questionnaire (REQ)
A total of 435 participants who attended the training course completed the PALAMA Reaction
Evaluation Questionnaire (REQ) which formed the basis of the participant analysis. This
questionnaire captured the perceptions of the participants regarding the training intervention
that they attended. The variables evaluated and reported on from the REQs are listed below:
the course content;
the facilitator;
the logistics; and
the overall results.
3.1.1 Course Content
3.1.1.1 Quantitative results
Participants completing the REQs were asked to rate three key aspects of the course content,
that is,
how practical the course content was in relation to their workplace;
the extent to which the materials were ”well organised”; and
the extent to which the training materials addressed their learning needs.
The rationale behind these questions was to get an understanding of the course content in
terms of its appropriateness (pitch), organisation of learning materials and its relevance. The
results of the 435 participant responses with regard to the course content are represented in
Table 3.1.
The quantitative and qualitative findings are presented in tabular format and each table contains
the results of all five modules.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
10
Table 3.1 Participant perceptions regarding the course content
Module Questions regarding the
course content
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (N=51)
How practical was the course content for your workplace?
92,7 5,1 1,1 1,1
How well was the material organised for the audience/ target group?
96,6 2,3 0 1,1
How did you find the learning materials?
96,6 2,3 0 1,1
Democratic governance and legislative process (N=77)
How practical was the course content for your workplace?
93,4 0,8 0 5,8
How well was the material organised for the audience/ target group?
88,9 5,3 0 5,8
How did you find the learning materials?
91,7 2,5 0 5,8
Media Communication (N=89)
How practical was the course content for your workplace?
96,9 3,1 0 0
How well was the material organised for the audience/ target group?
94,1 5,9 0 0
How did you find the learning materials?
93,6 4,4 0,9 1,1
Value based leadership for decision making (N=88)
How practical was the course content for your workplace?
94,5 3,8 0 1,7
How well was the material organised for the audience/ target group?
83,4 13,9 1 1,7
How did you find the learning materials?
85,2 11,3 0,9 2,6
Action Research for the legislature process (N=130)
How practical was the course content for your workplace?
98,5 0,5 1 0
How well was the material organised for the audience/ target group?
85,2 9,3 5,5 0
How did you find the learning materials?
84,3 8,6 4,1 3
(N=435)
The information in Table 3.1 displays the responses of the participants to the REQ that was
administered after the different training interventions. From Table 3.1 it is evident that between
92,7% and 98,5% of the respondents rated the practical nature of the course content to their
working environment as positive across the five modules (i.e. ratings of “excellent” and “good”
categories combined). Between 0,5% and 5,1% of the respondents rated the course content as
“average” and a negligible proportion of the participants (between 0% and 1,1%) rated it as
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
11
“poor”. These responses indicate that there was a high satisfaction level regarding the practical
nature of the training course in relation to the workplace of the participants.
With regard to the organisation of the learning materials (structure), between 83,4% and
96,6% of the respondents rated the organisation of the learning materials as positive (i.e.
“excellent” and “good” categories combined), while between 2,3% and 13,9% of the
respondents rated the organisation of the learning materials as “average”. A small proportion of
the respondents (between 0% and 5,5%) rated it as “poor”. These responses indicate high
satisfaction levels with regard to the organisation of the learning materials.
The last aspect in Table 3.1 is a representation of responses regarding participants’ perceptions
of the learning materials. Between 84,3% and 96,6% of the respondents had a positive
perception of the learning materials (i.e. “excellent” and “good” categories combined). The range
of “average” responses was between 2,3% and 11,3% while the “poor” responses ranged
between 0% and 4,1%.
These responses indicate high satisfaction levels with the learning materials.
3.1.1.2 Qualitative results
There were four hundred and thirty five (N=435) responses to the participant REQ survey. The
responses from the participants are summarised in Table 3.2.
Table 3.2 Participant comments regarding the course content
Module Structured survey
question Participant Responses
Information Communication Technology (ICT) Tools
What other comments do you have about the content?
The following comments support the positive quantitative
trends already reported, e.g.:
"I can use material every day for reference."
"Now I know how to operate a computer and I also know
how to access the internet."
The respondents recommended the following:
More practical engagement with the theory will assist a
better understanding of the learning content.
Democratic governance and legislative process
What other comments do you have about the
The following comments support the positive quantitative
trends already reported, e.g.:
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
12
Module Structured survey
question Participant Responses
content? "They have capacitated us as members of Parliament.”
The respondents recommended the following:
The notes of the facilitator should be provided to
participants.
Media Communication
What other comments do you have about the content?
The following comments support the positive quantitative
trends already reported, e.g.:
"The course is practical in nature and we had a lot of
sharing opportunities."
"The content was relevant to the work place."
The respondents recommended the following:
Text books should be prescribed as resource materials
for this course.
The course should make use of voice recordings to
enable improvement of participants’ public speaking.
Value based leadership for decision making
What other comments do you have about the content?
The following comments support the positive quantitative
trends already reported, e.g.:
“Very relevant to what I am doing."
The respondents recommended the following:
More research is needed on the topic to improve the
learning material.
Action Research for the legislature process
What other comments do you have about the content?
The following comments support the positive quantitative
trends already reported, e.g.:
"Excellently presented and relevant to my workplace."
"Everything in the course content was relevant and
informative."
The respondents recommended the following:
Pre-course material should be provided to participants.
3.1.2 The Facilitator
3.1.2.1 Quantitative results
Participants who completed the REQs were asked to rate the facilitators of their training course
against three criteria, that is,
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
13
how well the training was presented;
the level of expertise of the facilitator; and
the extent to which the facilitator demonstrated up to date knowledge of the public service.
Table 3.3 illustrates the responses to the questions relating to how well the course was
facilitated, as well as the extent of the facilitators’ expertise.
Table 3.3 Participant perceptions regarding the facilitation
Module Questions regarding
facilitation
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (N=51)
How well was the course facilitated/presented?
95,5 3,4 0 1,1
How do you regard the facilitator’s/presenter’s expertise?
98,9 0 0 1,1
Democratic governance and legislative process (N=77)
How well was the course facilitated/presented?
95 0 0 5
How do you regard the facilitator’s/presenter’s expertise?
92,6 0,8 0 6,6
Media Communication (N=89)
How well was the course facilitated/presented?
98,4 1,6 0 0
How do you regard the facilitator’s/presenter’s expertise?
98,4 1,6 0 0
Value based leadership for decision making (N=88)
How well was the course facilitated/presented?
92 5 0 3
How do you regard the facilitator’s/presenter’s expertise?
92 5 0 3
Action Research for the legislature process (N=130)
How well was the course facilitated/presented?
95,7 3,3 1 0
How do you regard the facilitator’s/presenter’s expertise?
95,5 1,9 1 1,6
(N=435)
From Table 3.3 it is evident that the majority of the respondents (between 92% and 98,4%)
rated the presentation skills of their facilitators as positive (i.e. ratings of “excellent” plus
“good” categories). Between 0% and 5% of the respondents rated the presentation skills of
their facilitators as “average” and a negligible proportion of the respondents (between 0% and
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
14
1%) rated it as “poor”. This response rate is indicative of very high satisfaction levels with the
presentation skills of facilitators used during the training of the Capacity Building Programme
for Members of Parliament and Provincial Legislatures.
A very large proportion of the respondents (between 92% and 98,9%) rated their facilitator’s
expertise as positive (i.e. ratings of “excellent” plus “good” categories). Between zero and five
percent (0% - 5%) of the respondents rated the level of expertise demonstrated by their
facilitator as “average” and a negligible number of the respondents (0% - 1%) rated it as “poor”.
These positive responses are indicative of very high satisfaction levels with the expertise levels
of facilitators used during the training of the Capacity Building Programme for Members of
Parliament and Provincial Legislatures.
In Table 3.4 below the respondents’ perceptions of whether the facilitators demonstrated up to
date knowledge of the public service are presented.
Table 3.4 Participant perceptions regarding the facilitators’ Public Service knowledge
Module Question
Participants Ratings (percentage)
Yes No Did not
Respond
Information Communication Technology (ICT) Tools (N=51)
Did the facilitator/presenter demonstrate updated knowledge of Public Service?
93,1 0 6,9
Democratic governance and legislative process (N=77)
Did the facilitator/presenter demonstrate updated knowledge of Public Service?
88,4 0 11,6
Media Communication (N=89) Did the facilitator/presenter demonstrate updated knowledge of Public Service?
83,9 0 16,1
Value based leadership for decision making (N=88)
Did the facilitator/presenter demonstrate updated knowledge of Public Service?
91 1,3 7,7
Action Research for the legislature process (N=130)
Did the facilitator/presenter demonstrate updated knowledge of Public Service?
89,1 1,8 9,1
(N=435)
A large percentage of the respondents (between 83,9% and 93,1%) indicated that the
facilitators who facilitated the Capacity Building Programme for Members of Parliament and
Provincial Legislatures demonstrated up to date knowledge of the Public Service. A
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
15
negligible proportion of the respondents (0% - 1,8%) did not agree with this statement. The
percentage of respondents who did not respond to this statement ranged between 6,9% and
16,1%.
3.1.2.2 Qualitative results
There were four hundred and thirty five (N=435) responses to the participant survey. The key
responses from the participants are recorded in Table 3.5.
Table 3.5 Participant perceptions regarding the facilitation of the course
Module Structured survey
question Participant Responses
Information Communication Technology (ICT) Tools
What other comments do you have about the facilitator?
The following comments support the positive quantitative
trends already reported, e.g.:
"He also demonstrated alternative ways of doing things
which are not in the study material. He is very energetic
and has a vast knowledge of the subject."
“Her interpersonal skills are very good and she does not
compromise on quality."
"She is knowledgeable and able to transfer skill without
being imposing."
The respondents did not make any recommendations
regarding the facilitator.
Democratic governance and legislative process
What other comments do you have about the facilitator?
The following comments support the positive quantitative
trends already reported, e.g.:
"He assisted learners to discover the knowledge instead of
telling them how to do it."
"The facilitator understands public service very well. This
includes how policies are made as well as implementation."
"He has made the course interesting and easy to follow."
The respondents recommended the following:
More time should be allowed for debate during class.
Media Communication
What other comments do you have about the facilitator?
The following comments support the positive quantitative
trends already reported, e.g.:
"She had a way of ensuring that we are always attentive
and that we participate actively.”
"The facilitator explained in detail and provided examples."
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
16
Module Structured survey
question Participant Responses
The respondents did not make any recommendations
regarding the improvement of programme facilitation.
Value based leadership for decision making
What other comments do you have about the facilitator?
The following comments support the positive quantitative
trends already reported, e.g.:
"The facilitator demonstrated vast knowledge of the subject
matter.”
“The facilitator is an expert."
“Very good interpersonal skills and professional."
The respondents did not make any recommendations
regarding the improvement of programme facilitation.
Action Research for the legislature process
What other comments do you have about the facilitator?
The following comments support the positive quantitative
trends already reported, e.g.:
"A vibrant and knowledgeable facilitator."
"The facilitator was on the level of the student. Every
person was given a chance to be involved in discussions."
"The facilitator has good knowledge of the subject and
allows interaction throughout."
The respondents did not make any recommendations
regarding the facilitators.
3.1.3 Logistics, venue and food
3.1.3.1 Quantitative results
Participants who completed the REQs were asked to rate the logistical arrangements
associated with their training programmes against three criteria, that is,
how well the training was organised;
whether the venue was conducive to learning; and
whether the facilities, food and accommodation were up to standard.
The survey results of the 435 respondents to the questions relating to how well the training was
organised, the conduciveness of the venue to learning and the standard of the facilities, food
and accommodation are represented in Table 3.6.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
17
Table 3.6 Participant perceptions regarding the logistics, venue and food
Module Questions regarding the
logistics, venue and food
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (N=51)
How well was the event organised?
89,8 7,4 0 2,8
Was the venue conducive for learning?
93,8 6,2 0 0
Were the facilities, food and accommodation up to standard?
75,8 17,5 0 6,7
Democratic governance and legislative process (N=77)
How well was the event organised?
88,4 2,1 0 9,5
Was the venue conducive for learning?
85 3,8 0 11,2
Were the facilities, food and accommodation up to standard?
81 7,8 0 11,2
Media Communication (N=89)
How well was the event organised?
87,2 7,7 0 5,1
Was the venue conducive for learning?
83,4 8,7 2,8 5,1
Were the facilities, food and accommodation up to standard?
73,7 16,4 3,3 6,6
Value based leadership for decision making (N=88)
How well was the event organised?
85,5 11,5 0 3
Was the venue conducive for learning?
89,5 8,8 0 1,7
Were the facilities, food and accommodation up to standard?
79,7 12,9 5,7 1,7
Action Research for the legislature process (N=130)
How well was the event organised?
88,9 6,8 2 2,3
Was the venue conducive for learning?
91,5 7,1 0 1,4
Were the facilities, food and accommodation up to standard?
77,5 17,9 2,3 2,3
(N=435)
From Table 3.6 it is evident that the majority of the respondents (between 85,5% and 89,8%)
rated the organisation of the training event as positive (i.e. rating of “excellent” and “good”
categories combined). A small proportion of the respondents (between 2,1% and 11,5%) rated
the organisation of the training event as “average” and a negligible proportion of them (between
0% and 2%) rated it as “poor”. These ratings are indicative that the respondents were satisfied
with the organisation of the training events.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
18
The second aspect which is dealt with in this table is the perceptions of the respondents
regarding the conduciveness of the training venue to learning. Between 83,4% and 93,8% of
the respondents felt that the training venues were conducive to learning (i.e. rating of “excellent”
and “good” categories combined). The percentage of respondents that rated the conduciveness
of the training venues as “average” ranged between 3,8% and 8,8% while a small percentage
rated them as “poor” (0 – 2,8%). These findings are indicative that the majority of the
participants were satisfied with the suitability of the training venues in terms of their
conduciveness to learning.
The last question in Table 3.6 indicates that between 73,7% and 81% of the respondents rated
the facilities, food and accommodation positively (i.e. rating of “excellent” and “good”
categories combined). A fair proportion of the respondents (between 7,8% and 17,9%) rated
the facilities, food and accommodation as ”average” and a small percentage (between 0% and
5,7%) rated these as “poor”.
3.1.3.2 Qualitative results
The key comments that emerged from this qualitative question with regard to the logistical
arrangements associated with programmes are summarised in Table 3.7.
Table 3.7 Participant comments regarding the logistics, venue and food
Module Structured survey
question Participant Responses
Information Communication Technology (ICT) Tools
What other comments do you have about the logistics of the course?
The following comments support the positive quantitative
trends already reported, e.g.:
“The venue was ideal for training.”
“Everything was well organized.”
The respondents recommended the following:
Pre-course reading material should be provided.
The notification period for course attendance should be
extended.
Democratic governance and legislative process
What other comments do you have about the logistics of the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"The logistics were perfect."
“Everything was in order.”
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
19
Module Structured survey
question Participant Responses
The respondents did not make any recommendations
regarding the improvement of Logistical arrangements.
Media Communication
What other comments do you have about the logistics of the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"Well organized.”
“The logistical arrangements were good.”
The respondents recommended the following:
The venues should accommodate physically
challenged participants.
The dietary requirements of participants should be
taken into consideration.
Value based leadership for decision making
What other comments do you have about the logistics of the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"The logistics has improved up to so far. There is a need
to keep it like that."
“The event was organized well."
The respondents recommended the following:
The programme material should be provided before the
training commences.
The variety of food should be improved.
Action Research for the legislature process
What other comments do you have about the logistics of the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"Logistics formed a valuable support framework for the
course."
“Excellent”.
"The venue owner was friendly and supportive. I really
appreciated it."
The respondents recommended the following:
Course material should be provided earlier to allow for
preparatory reading.
Note pads should be provided to participants.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
20
3.1.4 Overall Results
3.1.4.1 Quantitative results
Participants who completed the REQs were asked to do an overall evaluation of three key
aspects of the programme, that is,
their overall perception of the training programme that they had undergone;
the relevance of the skills they were taught to their work environment; and
the networking opportunities provided as a result of attending the training programme.
The REQ survey results of the 435 responses to these key aspects are represented in
Table 3.8.
Table 3.8 Overall perceptions of Participants
Module Questions regarding overall
perceptions of the Module
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (N=51)
Overall, how would you rate the course/event?
89,8 4,6 1,1 4,4
How do you rate the relevance of the course/event to your work?
93,2 1,1 1,1 4,5
How would you rate the course/event in networking opportunities provided?
94,4 1,1 0 4,5
Democratic governance and legislative process (N=77)
Overall, how would you rate the course/event?
95 0 0 5
How do you rate the relevance of the course/event to your work?
92,1 2,1 0 5,8
How would you rate the course/event in networking opportunities provided?
91,8 2,4 0 5,8
Media Communication (N=89)
Overall, how would you rate the course/event?
97,3 2,7 0 0
How do you rate the relevance of the course/event to your work?
97,3 2,7 0 0
How would you rate the course/event in networking opportunities provided?
92,3 5,2 0,8 1,7
Value based leadership for
Overall, how would you rate the course/event?
93,5 3,7 0 2,8
Poor
Poor
Poor
Poor
Poor
Poor
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
21
Module Questions regarding overall
perceptions of the Module
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
decision making (N=88)
How do you rate the relevance of the course/event to your work?
95,8 2,5 0 1,7
How would you rate the course/event in networking opportunities provided?
91 6,3 1 1,7
Action Research for the legislature process (N=130)
Overall, how would you rate the course/event?
85,8 0,5 1,0 12,7
How do you rate the relevance of the course/event to your work?
85,3 1 1 12,7
How would you rate the course/event in networking opportunities provided?
78 6 1,4 14,6
(N=435)
The responses to the first question in Table 3.8 show that a large percentage of the
respondents (85,8% - 97,3%) rated the training course positively (i.e. “excellent” and “good”
categories combined). A small proportion of the respondents (0% - 4,6%) rated the training
course as “average” and between 0% and 1,1% of the participants rated the course as “poor”.
These responses indicated very high satisfaction levels with the training course that was
offered.
It is evident from the second question in Table 3.8 that between 85,3% and 97,3% of the
respondents rated the relevance of the training course to their work as positive (i.e. ratings
of “excellent” and “good” categories combined). A small percentage of respondents (between
1% and 2,7%) rated the relevance of the training programme as “average”. Between 0% and
1,1% of the participants rated it as “poor”. These responses indicate high satisfaction levels
with the relevance of the training course to the work environment of the respondents.
The percentage of respondents that rated the networking opportunities provided as a result of
attending the training programme as positive (i.e. ratings of “excellent” and “good” categories
combined) ranged between 78% and 94,4%. It should be noted that between 1,1% and 6,3% of
the respondents rated the networking opportunities provided as a result of attending the training
programme as ”average” and a small percentage of the respondents (between 0% and 1,4%)
Poor
Poor
Poor
Poor
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
22
rated it as “poor”. Therefore it is evident from Table 3.8 that most of the respondents perceived
the training intervention as a networking opportunity.
3.1.4.2 Qualitative results
The key recommendations that emerged on participants’’ overall rating of the programme are
summarised in Table 3.9.
Table 3.9 Participant comments regarding the overall rating of the course
Module Structured survey
question Participant Responses
Information Communication Technology (ICT) Tools
What other comments do you have about the course?
The following comments support the positive quantitative
trends already reported, e.g.:
“This was a good opportunity to sharpen my skills."
“The course material was well prepared and relevant.”
"It was quite an empowering course especially with regard
to internet and e-mail."
The respondents recommended the following:
The course needs to be extended.
Groups need to be composed of individuals with the
same level of competency.
Democratic governance and legislative process
What other comments do you have about the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"The course is relevant to us as MPs and it supports us to
conduct our oversight role."
"The course has been an eye opener to me regarding
matters of budgeting. This is good for my work as a public
representative."
The respondents recommended the following:
Conflict management and the grievance procedure
should be included in the course.
The course should contain more practical work.
Media Communication
What other comments do you have about the course?
The following comments support the positive quantitative
trends already reported, e.g.:
"High quality content. Well researched."
"I feel more confident to face the media now."
"The course is very good because it takes your fear of the
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
23
Module Structured survey
question Participant Responses
media away."
The respondents recommended the following:
The course should include how to prepare a media
statement.
The media presentation should be recorded so that it
can be available to participants after the course.
Value based leadership for decision making
What other comments do you have about the course?
The following comments support the positive quantitative
trends already reported, e.g.:
“Very good course to remind us about our responsibility as
public representatives."
“The course enhanced the knowledge that we had for the
purpose of doing community work."
The respondents recommended the following:
The facilitator need to do more research on the topic
and provide more practical examples.
Action Research for the legislature process
What other comments do you have about the course?
The following comments support the positive quantitative
trends already reported, e.g.:
“The action research course is extremely useful in the
legislative environment."
"It was informative and facilitated with passion. It is 100%
relevant. It is excellent."
The respondents recommended the following:
Learning material should be provided in advance to
enable learners to prepare.
Participants need more opportunity to interact on
issues raised.
Handouts should be made available in class rather
than having it e-mailed after the course.
Communication regarding course dates should be
made available at least a month in advance.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
24
3.2 Facilitator Survey: Facilitators Feedback Form (FFF)
A total of 23 Facilitator Feedback questionnaires (out of the 25 training interventions presented)
were received which represented the facilitators’ perceptions of the course that they facilitated.
The PALAMA Facilitator Feedback questionnaire formed the basis of the facilitator analysis
conducted. The variables evaluated and reported on from the Facilitator Feedback Form (FFF)
are listed as follows:
Course content
Participants
Facilitation
Logistics
Recommendations
3.2.1 Course Content
3.2.1.1 Quantitative results
Facilitators who completed the Facilitator Feedback Form (FFF) were asked to rate three key
aspects of the programme content, that is,
Is the course/event the right length?
Is the course material paced appropriately for the audience/target group?
Does the course material relate sufficiently to real public sector or situation experienced by
the audience/target group?
The rationale behind these questions was to get an understanding of the facilitators’ perceptions
of the course content regarding the length of the course, the pacing of the course materials and
relevance of the learner materials to the public sector from which participants came. The results
of the responses from the facilitators to the above mentioned questions are represented in
Tables 3.10, 3.11 and 3.12 that follow.
Please note that the number of training interventions per module has been indicated in brackets
in the tables.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
25
Table 3.10 Facilitator perceptions regarding the length of the course
Module
Question regarding
the length of the
course
Participants Ratings (percentage)
Too much Appropriate Could be
more
Did not
respond
Information Communication Technology (ICT) Tools (presented 3 times)
Is the course/event the right length?
0 100 0 0
Democratic governance and legislative process (presented 4 times)
Is the course/event the right length? 0 50 50 0
Media Communication (presented 6 times)
Is the course/event the right length?
0 83,3 16,7 0
Value based leadership for decision making (presented 5 times)
Is the course/event the right length? 0 100 0 0
Action Research for the legislature process (presented 7 times)
Is the course/event the right length? 0 57,1 14,3 28,6
Facilitator feedback forms were received for 23 of the 25 training interventions.
Table 3.10 indicates that the majority of the facilitators (between 57,1% and 100%) thought the
length of the course was “appropriate” in relation to the content that had to be covered with
reference to four of the five modules. Two of the four facilitators (50%) of the “Democratic
governance and legislative process” module of the programme indicated that its length could be
increased in relation to the content that needed to be covered while for the remaining four
modules the percentages range between 0% and 16,7%.
Table 3.11 illustrates the perception of the facilitators regarding the pacing of the course
material.
Table 3.11 Facilitator perceptions regarding the pacing of the course materials
Module
Question regarding the
pacing of the course
material
Participants Ratings (percentage)
Too Quick Appropriate Too slow Did not
respond
Information Communication Technology (ICT) Tools (presented 3 times)
Is the course material paced appropriately for the audience/target group?
0 100 0 0
Democratic governance and legislative process
Is the course material paced appropriately for
25 75 0 0
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
26
Module
Question regarding the
pacing of the course
material
Participants Ratings (percentage)
Too Quick Appropriate Too slow Did not
respond
(presented 4 times) the audience/target group?
Media Communication (presented 6 times)
Is the course material paced appropriately for the audience/target group?
0 100 0 0
Value based leadership for decision making (presented 5 times)
Is the course material paced appropriately for the audience/target group?
0 80 0 20
Action Research for the legislature process (presented 7 times)
Is the course material paced appropriately for the audience/target group?
14,3 57,1 0 28,6
(N=25)
It is evident in Table 3.11 that between 57,1% and 100% of the facilitators thought the pace of
the programme was “appropriate” in relation to content that needed to be covered while
between 0% and 25% thought pacing was “too quick” and none of the facilitators indicated that
the pace was too slow. It should be noted that the percentage of facilitators who did not
respond, ranged between 0% and 28,6%.
These responses indicate that the majority of the facilitators were of the opinion that an
appropriate pace was used to cover the content in the course.
Table 3.12 Facilitator perceptions regarding the relevance of the materials to the Public Sector
Module
Question regarding the
relevance of the materials
to the Public Sector
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (presented 3 times)
Does the course material relate sufficiently to real public sector situation experienced by the audience/ target group?
100 0 0 0
Democratic governance and legislative process (presented 4 times)
Does the course material relate sufficiently to real public sector situation experienced by the audience/ target group?
50 50 0 0
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
27
Module
Question regarding the
relevance of the materials
to the Public Sector
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Media Communication (presented 6 times)
Does the course material relate sufficiently to real public sector situation experienced by the audience/ target group?
83,3 0 0 16,7
Value based leadership for decision making (presented 5 times)
Does the course material relate sufficiently to real public sector situation experienced by the audience/ target group?
80 20 0 0
Action Research for the legislature process (presented 7 times)
Does the course material relate sufficiently to real public sector situation experienced by the audience/ target group?
57,1 14,3 0 28,6
(N=25)
In Table 3.12 it is evident that between 50% and 100% of the facilitators were of the opinion that
the course materials were very well related to the Public Sector (i.e. ratings of “excellent” and
“good” categories combined). Between 0% and 50% thought the course materials have an
“average” rating with regard to their relatedness to the Public Sector, while none of the
facilitators felt the course material related poorly to the Public Sector.
These responses indicate that with reference to four of the five modules a large proportion of
the facilitators surveyed were of the opinion that the course materials related sufficiently to the
public sector.
3.2.1.2 Qualitative results
The key responses from the facilitator survey are summarised in Table 3.13.
Table 3.13 Facilitator comments regarding the course content
Module Structured survey
question
Facilitator responses
Information Communication Technology (ICT) Tools
Are there any
recommendations or
comments with regard to
The fact that participants use different computer operating
systems complicates the training.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
28
the content or layout you
would like to make?
Democratic governance and legislative process
Are there any
recommendations or
comments with regard to
the content or layout you
would like to make?
The section on policy must cover analysis as well as
monitoring and evaluation.
The budgeting content must be geared towards the
participants’ oversight role and should include the analysis
of Provincial/Departmental statistics.
The learning material is not sufficient and was
supplemented by the facilitators. More case studies are
needed as well as more emphasis on processes.
The materials redesign process should take into
consideration that law-making is the centre of MPs work.
The pitch should be appropriate.
Media Communication
Are there any
recommendations or
comments with regard to
the content or layout you
would like to make?
The facilitator should be allowed a follow up session with
participants to ascertain whether the learning content is
applied in practice. This process can also inform the
reworking of the learning material.
The learning content should also include the rationale for the
communication process as well as a description of the
different media scenarios which could be encountered.
More practical exercises should be included in the learning
material.
The learning material utilised on the programme from 25 –
27 January 2011 in Mpumalanga had some chapters
missing.
Value based leadership for decision making
Are there any
recommendations or
comments with regard to
the content or layout you
would like to make?
The learning material should include more relevant
references to politicians rather than public officials in
general.
The current learning material is supplemented with
additional material by facilitators.
Action Research for the legislature process
Are there any
recommendations or
comments with regard to
the content or layout you
would like to make?
Case studies which are relevant to the legislative process
should be included.
The learning material is insufficient and not sequenced in a
logical manner. A detailed suggestion regarding reworking
of the programme was received.
3.2.2 Facilitator Assessment of Participants
3.2.2.1 Quantitative results
The responses of the facilitators surveyed regarding the appropriateness of the
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
29
level of the participants who attended the course;
number of the participants per training intervention;
mix of the participants; and
trainees attendance and participation
are displayed in Tables 3.14 – 3.17 that follow.
Table 3.14 Facilitator perceptions regarding the appropriate level of the participants
Module
Question regarding the
appropriate level of the
participants
Participants Ratings (percentage)
Too High Appropriate Too Low Did not
respond
Information Communication Technology (ICT) Tools (presented 3 times)
Were the participants at the appropriate level for the course/event?
0 66,7 0 33,3
Democratic governance and legislative process (presented 4 times)
Were the participants at the appropriate level for the course/event?
0 100 0 0
Media Communication (presented 6 times)
Were the participants at the appropriate level for the course/event?
0 100 0 0
Value based leadership for decision making (presented 5 times)
Were the participants at the appropriate level for the course/event?
0 100 0 0
Action Research for the legislature process (presented 7 times)
Were the participants at the appropriate level for the course/event?
0 71,4 0 28,6
(N=25)
From Table 3.14 it is evident that between 66,7% and 100% of the facilitators thought the
participants were at an “appropriate level” in relation to the complexity of the course materials
that needed to be covered. No facilitators felt that the level of participants were either “too high”
or “too low”. In the two instances where facilitators did not select the “appropriate” response
they did not respond to the survey question.
Table 3.15 is a representation of the perceptions of the facilitators regarding the
appropriateness of the number of participants who attended the training interventions.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
30
Table 3.15 Facilitator perceptions regarding the number of participants who attended
Module
Question regarding the
number of participants
who attended
Participants Ratings (percentage)
Too Many Appropriate Too Few Did not
respond
Information Communication Technology (ICT) Tools (presented 3 times)
Was the number of participants too few, too many or in the right zone for this course/event?
0 66,7 0 33,3
Democratic governance and legislative process (presented 4 times)
Was the number of participants too few, too many or in the right zone for this course/event?
0 75 0 25
Media Communication (presented 6 times)
Was the number of participants too few, too many or in the right zone for this course/event?
0 100 0 0
Value based leadership for decision making (presented 5 times)
Was the number of participants too few, too many or in the right zone for this course/event?
0 100 0 0
Action Research for the legislature process (presented 7 times)
Was the number of participants too few, too many or in the right zone for this course/event?
0 57,1 14,3 28,6
(N=25)
From Table 3.15 it is evident that between 57,1% and 100% of the facilitators thought that the
number of participants who attended the training was “appropriate”, while in the case of the
“Action Research for the legislature process” module, 14,3% of the facilitators thought that there
were “too few” participants who attended the training interventions.
These responses indicate that the majority of the facilitators were of the opinion that the number
of participants who attended the training intervention was appropriate.
Table 3.16 is a representation of the facilitators’ perception of whether the mix of participants
was appropriate.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
31
Table 3.16 Facilitator perceptions regarding the mix of participants
Module Question regarding the
mix of participants
Participants Ratings (percentage)
Not
uniform
enough
Appropriate
mix
Too
uniform
Did not
respond
Information Communication Technology (ICT) Tools (presented 3 times)
Was the mix of participants appropriate?
33,3 66,7 0 0
Democratic governance and legislative process (presented 4 times)
Was the mix of participants appropriate? 25 75 0 0
Media Communication (presented 6 times)
Was the mix of participants appropriate?
0 83,3 16,7 0
Value based leadership for decision making (presented 5 times)
Was the mix of participants appropriate? 0 80 20 0
Action Research for the legislature process (presented 7 times)
Was the mix of participants appropriate? 14,2 28,6 28,6 28,6
(N=25)
From Table 3.16 it is evident that between 28,6% and 83,3% of the facilitators thought that the
mix of participants who attended the training was “appropriate”. The percentage of facilitators
who rated the mix of participants as “not uniform enough” ranged between 0% and 33,3% while
between 0% and 28,6% of facilitators rated the mix of participants as “too uniform”. These
responses indicate that according to the majority of the facilitators (with the exception of the
“Action Research for the legislature process” module), the mix of participants who attended the
Capacity Building Programme for Members of Parliament and Provincial Legislatures was well
balanced.
Table 3.17 is a representation of the facilitators’ perception of whether the attendance and
participation of the participants was appropriate.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
32
Table 3.17 Facilitator perceptions regarding the attendance of the participants
Module
Question regarding the
attendance of the
participants
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (presented 3 times)
Was the attendance and participation appropriate?
66,7 33,3 0 0
Democratic governance and legislative process (presented 4 times)
Was the attendance and participation appropriate? 100 0 0 0
Media Communication (presented 6 times)
Was the attendance and participation appropriate? 100 0 0 0
Value based leadership for decision making (presented 5 times)
Was the attendance and participation appropriate? 80 0 0 20
Action Research for the legislature process (presented 7 times)
Was the attendance and participation appropriate? 42,8 28,6 0 28,6
(N=25)
From Table 3.17 it is evident that with the exception of the “Action Research for the legislature
process” module the majority of the facilitators (between 66,7% and 100%) thought that the
attendance and participation of trainees who attended the programme was positive (i.e.
ratings of “excellent” and “good” categories combined). With reference to the “Action Research
for the legislature process” module, 28,6% of the facilitators rated the attendance and
participation of trainees as “average” while a further 28,6% of facilitators did not respond to this
question. An “average” percentage of 33,3% was recorded in relation to the Information
Communication Technology course. None of the facilitators rated the attendance and
participation of trainees as “poor”.
Although this is a very positive response, there is still room for improvement regarding the
attendance rate of the participants and their commitment to the training specifically because this
programme is based on the notion of action learning.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
33
3.2.2.2 Qualitative results
The facilitators noted certain concerns in respect of the attendance of the participants.
Table 3.18 Facilitator comments regarding the attendance of the participants
Module Structured survey
question
Facilitator responses
Information Communication Technology (ICT) Tools
Are there any
recommendations with
regards to participants which
you would like to make?
Members should respect the facilitators during
training e.g. refrain from leaving the classroom,
answering cell phones during class as well as having
conversations during class.
Democratic governance and legislative process
Are there any
recommendations with
regards to participants which
you would like to make?
Despite the fact that MPs are on call the participation
was good.
Media Communication
Are there any
recommendations with
regards to participants which
you would like to make?
The percentage of attendees versus the number of
members registered for the training intervention is
low.
There should be an attempt to improve class
attendance during the training.
The scheduling of the training intervention has a
direct influence on the availability of participants due
to their planned work commitments in
Parliament/Provincial Legislatures.
Value based leadership for decision making
Are there any
recommendations with
regards to participants which
you would like to make?
Participants should be more punctual.
The participants were of a high level and they
enthusiastically contributed to the discussions.
Group sizes should be increased.
The marketing of the programme should be
improved.
Action Research for the legislature process
Are there any
recommendations with
regards to participants which
you would like to make?
Participants should be more punctual.
The marketing should be improved with specific
reference to the value of the programme.
Participants were absent frequently for official duties.
Learning was very interactive and the participants
were very co-operative and open-minded.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
34
3.2.3 Facilitation
Since it would be very subjective to ask facilitators to rate their own performance as facilitators,
they were asked two respond to two qualitative questions regarding their perceptions of their
facilitation of the course. The questions were:
Are there elements of your presentation/ facilitation that you would like to pass on to other
facilitators?
Are there elements of your presentation/ facilitation that you would do differently next
time?
Facilitator responses to each of these questions above are illustrated in Table 3.19.
Table 3.19 Facilitator comments regarding the facilitation
Module Structured survey
question
Facilitator responses
Information Communication Technology (ICT) Tools
Are there elements of your
presentation/ facilitation that
you would like to pass on to
other facilitators?
Are there elements of your
presentation that you would
do differently next time?
PALAMA should provide 3G cards to enable practical
demonstrations pertaining to the internet and e-mail.
An analysis of the target group in terms of their
relevant knowledge/skill levels should be done before
the module commences.
Democratic governance and legislative process
Are there elements of your
presentation/ facilitation that
you would like to pass on to
other facilitators?
Are there elements of your
presentation that you would
do differently next time?
Several facilitators have developed additional
material which they used during facilitation.
Media Communication
Are there elements of your
presentation/ facilitation that
you would like to pass on to
other facilitators?
Are there elements of your
presentation that you would
do differently next time?
This module requires an interactive style and a great
deal of participation by the attendees.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
35
Module Structured survey
question
Facilitator responses
Value based leadership for decision making
Are there elements of your
presentation/ facilitation that
you would like to pass on to
other facilitators?
Are there elements of your
presentation that you would
do differently next time?
“Tuning in” exercises should be utilised to get
participants thinking before facilitation starts.
Facilitators should get together to share experiences.
Action Research for the legislature process
Are there elements of your
presentation/ facilitation that
you would like to pass on to
other facilitators?
Are there elements of your
presentation that you would
do differently next time?
The introduction of an informal pre-test would add
value.
Group work and group reports should be
incorporated into the module.
It was useful to refer to relevant incidents which are
currently receiving national prominence e.g. service
delivery protests, housing delivery etc. and to use
these as examples for action research.
The time allocated to the complete the module
should be extended to allow the assessment of
participants.
3.2.4 Logistics
3.2.4.1 Quantitative results
The various responses to the facilitator survey which referred to their perception of the
organisation of the course, the suitability of the venue for training and the standard of the
facilities, food and accommodation are illustrated in Tables 3.20 – 3.22.
Table 3.20 Facilitator perceptions regarding the organisation of the course
Module Question regarding the
organisation of the course
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (presented 3 times)
How well was the course/event organised?
66,7 33,3 0 0
Democratic How well was the 100 0 0 0
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
36
Module Question regarding the
organisation of the course
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
governance and legislative process (presented 4 times)
course/event organised?
Media Communication (presented 6 times)
How well was the course/event organised? 83,3 16,7 0 0
Value based leadership for decision making (presented 5 times)
How well was the course/event organised? 80 0 20 0
Action Research for the legislature process (presented 7 times)
How well was the course/event organised? 57,1 14,3 0 28,6
(N=25)
From Table 3.20 it is evident that between 57,1% and 100% of the facilitators thought that the
organisation of the course was well done (i.e. ratings of “excellent” and “good” categories
combined). Furthermore between 0% and 33,3% of the respondents were of the opinion that
the organisation of the course was “average”, while one of the five facilitators of the “Value
based leadership for decision making” module (20%) thought that it was “poor”.
These responses indicate that, although there is a high positive response, the organisation of
the training course can still be improved.
Table 3.21 is a representation of the facilitators’ perception of whether the training venues were
suitable for facilitation.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
37
Table 3.21 Facilitator perceptions regarding the suitability of the training venue
Module
Question regarding the
suitability of the training
venue
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (presented 3 times)
Was the venue favourable for teaching?
66,7 33,3 0 0
Democratic governance and legislative process (presented 4 times)
Was the venue favourable for teaching? 100 0 0 0
Media Communication (presented 6 times)
Was the venue favourable for teaching? 66,7 33,3 0 0
Value based leadership for decision making (presented 5 times)
Was the venue favourable for teaching? 80 20 0 0
Action Research for the legislature process (presented 7 times)
Was the venue favourable for teaching? 71,4 0 0 28,6
(N=25)
Between 66,7% and 100% of the facilitators responded positively to the question whether the
venue is conducive for facilitation (i.e. ratings of “excellent” and “good” categories
combined). Between 0% and 33,3% of facilitators thought that it was “average” and none of the
facilitators thought that the suitability of the training venue for facilitation was “poor”.
These responses indicate that according to facilitators, the venues selected for hosting the
Capacity Building Programme for Members of Parliament and Provincial Legislatures are
conducive for facilitation.
Figure 3.22 is a representation of the facilitators’ perception of whether the facilities, food and
accommodation were up to standard.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
38
Table 3.22 Facilitator perceptions regarding the facilities, food and accommodation
Module
Question regarding the
facilities, food and
accommodation
Participants Ratings (percentage)
Positive Rating
(ratings of
“excellent” and
“good” categories
combined)
Average Poor Did not
Respond
Information Communication Technology (ICT) Tools (presented 3 times)
Were the facilities, food and accommodation up to standard? 100 0 0 0
Democratic governance and legislative process (presented 4 times)
Were the facilities, food and accommodation up to standard?
100 0 0 0
Media Communication (presented 6 times)
Were the facilities, food and accommodation up to standard?
66,7 33,3 0 0
Value based leadership for decision making (presented 5 times)
Were the facilities, food and accommodation up to standard?
80 20 0 0
Action Research for the legislature process (presented 7 times)
Were the facilities, food and accommodation up to standard?
57,1 14,3 0 28,6
(N=25)
From Table 3.22 it is evident that between 57,1% and 100% of the facilitators positively
perceived the facilities, food and accommodation (i.e. ratings of “excellent” and “good”
categories combined). The percentage of facilitators who indicated that the facilities, food and
accommodation were “average” ranged between 0% and 33,3%. No facilitators rated the
facilities, food and accommodation as “poor”.
These responses indicate that according to facilitators, the facilities, food and accommodation
at the venues selected for hosting the Capacity Building Programme for Members of Parliament
and Provincial Legislatures are acceptable.
It has to be mentioned that one facilitator who presented two of the “Action research for
the legislature process” modules did not complete the required Facilitator Feedback
Form. This resulted in a chronic “did not respond” rate of 28,6% with reference to all the
tables above that summarise the quantitative facilitator feedback.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
39
3.2.4.2 Qualitative Results
The key recommendations that emerged from the facilitator survey are summarized in
Table 3.23.
Table 3.23 Facilitator comments regarding the Logistics
Module Structured survey
question
Facilitator responses
Information Communication Technology (ICT) Tools
How could the logistics be
improved? Please motivate.
Facilitators should be informed of the venues
timeously.
The seating arrangement was not optimal in certain
instances.
Democratic governance and legislative process
How could the logistics be
improved? Please motivate.
Facilitators should be informed of the venues
timeously.
The availability of projectors should be improved.
In some instances not enough learning materials were
available for all participants.
Media Communication
How could the logistics be
improved? Please motivate.
The module should be presented away from the
Legislature.
Technical support e.g. laptop and projector was
lacking.
Value based leadership for decision making
How could the logistics be
improved? Please motivate.
Venues should have flexible seating arrangements.
Break-away rooms should be available.
Action Research for the legislature process
How could the logistics be
improved? Please motivate.
The seating arrangement should be flexible in the
training venue.
3.2.5 Facilitators’ suggestions regarding the improvement of the Capacity Building
Programme for Members of Parliament and Provincial Legislatures
The key themes that emerged from the feedback forms are summarized in Table 3.24.
Table 3.24 Facilitator suggestions regarding the improvement of the Capacity Building
Programme for Members of Parliament and Provincial Legislatures
Module Structured survey
question
Facilitator responses
Information Communication Technology (ICT) Tools
What other suggestions
could help the Academy
improve the course?
Members should be categorised in accordance with
their levels of knowledge per group to ensure
maximum transfer of learning in the allocated time
per course.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
40
Module Structured survey
question
Facilitator responses
Democratic governance and legislative process
What other suggestions
could help the Academy
improve the course?
Facilitators should be involved in the materials
development process.
Media Communication
What other suggestions
could help the Academy
improve the course?
Formal assessments should be introduced into the
module.
The learning material should arrive well in time at the
respective training venues.
Presentation of modules should be scheduled better.
Members could benefit from private coaching in
public speaking.
Value based leadership for decision making
What other suggestions
could help the Academy
improve the course?
The training manual and slides have to be updated.
The facilitation style should be adapted based on the
group composition and circumstances of the specific
legislature.
Learning material should be dispatched on time.
Action Research for the legislature process
What other suggestions
could help the Academy
improve the course?
This programme should be presented soon after the
elections so that members can be empowered to
contribute maximally for their full term in the
legislature.
Participants should read the learning material before
the module commences.
This module requires more time.
This concludes the findings of the evaluation. Chapter 4 deals with the recommendations and
conclusion of the evaluation.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
41
CHAPTER 4
RECOMMENDATIONS AND CONCLUSION
4.1 INTRODUCTION
This evaluation report is based on an analysis of the feedback received from the participants
and the facilitators who participated in the Capacity Building Programme for Members of
Parliament and Provincial Legislatures. In Chapter 3 the data was analysed in detail according
to the quantitative and qualitative responses received. The key recommendations are
summarised in this chapter.
4.2 RECOMMENDATIONS
Based on the perceptions of the facilitators and the participants the following recommendations
are made:
The learning material for the programme should be revised in cooperation with the
facilitators with attention to the following:
the use of additional reference material where applicable;
logical sequencing of the material;
the use of relevant practical exercises and case studies to strengthen the learning
experience;
customisation of the learning material to the unique training needs of Members of
Parliament and Provincial Legislatures.
The nomination of participants should be done timeously (participants mentioned one month
as a guideline).
The ratio of members nominated versus those who attend the programme should be
monitored to ensure optimal utilisation of every training intervention.
The participants should be subjected to a summative assessment at the completion of each
module of the programme.
The profiles of the participants should be provided to the facilitators prior to the training so
that the facilitators can prepare adequately.
The participants should receive the learning material beforehand to be able to prepare
adequately.
The training venues should be user friendly to people with disabilities.
Capacity Building Programme for Members of Parliament and Provincial Legislatures: May 2011
42
The programme should be extensively marketed to ensure that the value of the programme
is understood.
A forum for the facilitators should be established to allow them to share their experiences
and best practices with each other as well as with PALAMA.
New members of Parliament and Provincial Legislatures should be provided with an
opportunity to attend this programme as soon as possible after appointment.
The feedback received from both the facilitator and participants should be forwarded to the
programme manager for each training intervention completed.
Onsite evaluations should be conducted on a sample of the training interventions.
4.3 CONCLUSION
The perceptions of the facilitators and the participants of the Capacity Building Programme for
Members of Parliament and Provincial Legislatures provided valuable inputs to enhance the
quality of the training intervention to ensure that the Members of Parliament and Provincial
Legislatures are fully capacitated and enabled to exercise their role and responsibilities.
It is important to note that the positive feedback received from both participants and facilitators
will not be used to retain the content and methodology of existing training programmes. The
review of PALAMA programmes is scheduled independently as part of the training cycle.
Although it takes evaluation findings into consideration, the programme review process looks at
all other relevant information.