Student Perceptions of Effective E-moderation: A Qualitative Investigation of E-College Wales
Brychan Thomas, Paul Jones, Gary Packham and Christopher Miller
University of Glamorgan
This study examines student perceptions of effective e-moderation and contrasts against existing literature. The study investigates the online BA Enterprise offered by the University of Glamorgan, supported by “Blackboard” software. The methodology involved interviews with 35 students using a semi-structured questionnaire to elicit responses on perceptions of effective e-moderation. Questionnaire analysis revealed that students identified quality and quick turnaround of feedback, availability of a tutor at key times and approachability as constituents of effective e-moderation. The e-moderator was characterised as a councillor, tutor and subject specialist. This study can be utilised by e-learning practitioners to develop an effective code of practice for e-moderators.
Student, effectiveness, characteristics, perception, e-moderation
The development and emergence of e-learning provides challenges for the providers of on-line courses in the development of effective pedagogy. A key facet to the emergent pedagogy is the role of the on-line tutor or e-moderator. The e-moderator faces a diversity of challenges including instructional design and organisation, facilitating discourse and directing student instruction (Anderson & Garrison, 1998). To achieve an understanding of what behaviour constitutes effective e-moderation it is critical that the student experience and perceptions of e-moderation are investigated. This investigation of the student experience will form the basis for the development of a code of practice for effective e-moderation within E-College Wales (ECW). This paper reports the results of a survey of student perceptions of effective emoderation. The aim of this study was to elicit responses as to what factors constituted effective e-moderation and the role of the on-line tutor. The research instrument asked respondents to identify examples of effective and ineffective e-moderation practice, characteristics of the e-moderator and suggestions for “best practice”.
Literature concerning student perceptions of effective e-moderation can be reviewed according to student perceptions and experience, student attitudes and satisfaction, student contributions and relationships, and course effectiveness. These are described below.
Student’s perceptions of distance learning, on-line learning and the traditional classroom were considered by O’Malley and McCraw (1999). They noted that until the late 1980’s, the primary educational delivery model for colleges was essentially the traditional lecture. Students were predominantly single, residential eighteen to twenty-three year olds although non-traditional working students were increasing. With the New Millennium a greater proportion of the student population is changing to married, employed and non-residential. With new technologies, knowledge delivery modules have also developed to better equip the educational sector in order that they can adequately respond to this market demand. These now include on-line education (with access through the Internet) and distance education involving interactive learning. From the study undertaken it was found that students perceive distance and on-line learning technologies to have provided some benefits. Pearl (2003) reports that online learning courses are aimed at those who prefer student centred learning which enables them to manage their time within course constraints. This gives students the means to study at home, in the office or on the move.
Picciano (2002) goes beyond student perceptions and considers issues of interaction, presence and performance in an online course. Questions exist on the nature and extent of the interaction and the effect on student performance. Much research is based on student perception of the quality and quantity of their interactions and how much they learn on an online course. The purpose of the study reported by Picciano (2002) was to examine online course performance in relationship to student interaction and the sense of presence on the course. The study attempted to go further than typical institutional performance measures such as grades and withdrawal rates and examined measures linked to course objectives. The results of a survey (Volery and Lord, 2000) concerning students enrolled on an on-line management course at an Australian University identified three critical success factors as technology, the instructor and the previous use of the technology from a student’s perspective.
Student experience
It is apparent that successful e-learning takes place within a system involving the student experience of e-learning and the teaching-learning context (Alexander, 2001). This conception of e-learning has a major influence on what students learn. A paper by Childs (2003) describes the ANNIE project (Accessing and Networking with National and International Expertise), which aimed to enhance student’s learning experience in Theatre Studies. This was achieved by enabling access to research-led teaching and to workshops led by practitioners of national and international standing. Most participating students had little or no experience of educational technologies and the participating university departments had not developed a distance-learning infrastructure. The project therefore initiated the development of expertise and technological support for remote access to experts in the departments.
Cox, Clark, Heath and Plumpton (2000) examined the online processes of 700 students’ experiences and made recommendations for good practice. They remark that an environment exists where technological change is daily and that the understanding of student learning online is in its infancy. Denard (2003) offers a reflection on experiences of designing Web activities that seek to develop student autonomy and creativity in learning as well as supporting student’s collaborative work. This is set in the context of resource-based learning, which takes place most effectively when students are engaged in creating, manipulating, editing and interpreting resources.
Students’ attitudes and satisfaction
Arbaugh (2001) has assessed how instructor immediacy behaviours affect student satisfaction and learning in Web-based MBA courses. In this study instructor classroom behaviours ‘immediacy behaviours’ were considered to determine whether they were significantly associated with student learning and satisfaction. It was noted that immediacy behaviours represent instructors’ attempts to reduce the social distance between themselves and students. The study found that while immediacy behaviours were positive predictors of student learning and satisfaction other factors such as student attitudes towards course software, length of course and prior student experience with Web-based courses were also significant predictors.
Wang (2003) comments that current models for measuring students’ evaluation of teaching effectiveness are perceived as inapplicable for measuring learner satisfaction with asynchronous e-learning systems since they are aimed at either organisational information systems or the classroom education environment. As a consequence his study developed a comprehensive model and instrument for measuring learner satisfaction.
Student contributions and relationships
Benfield (2000) notes that the teacher is at the behest of the students’ actions (or lack of them) when moving from face-to-face teaching to online delivery. As a consequence student passivity or ‘silence’ is difficult to interpret. Accordingly there is a need to establish a welcoming, ‘safe’ environment that encourages students to contribute to written discussions. By using student self-assessment it is possible to create a learning environment that is characterised as more student-centred than teacher centred (Berge, 1996). Evidence shows that there is a long history in the Open University and its Business School of a social constructivist approach to distance education emphasising a facilitative pedagogical style encouraging dialogue to, and from, the student at a distance (Salmon and Giles, 1997).
Coppola, Hiltz and Rotter (2002) describe the affective role of e-learning which relates to relationships with students and requires new tools to express emotion while making the relationship more intimate. This is in contrast with the managerial role, which deals with student monitoring. They report a change in the teaching persona emphasizing multilogues with students.
Course effectiveness
Drago, Peltier and Sorenson (2002) form measures of quality of on-line course content by using a standard student evaluation questionnaire. They note that there are a growing number of studies, which have been directed at programmes where an instructor is teaching students simultaneously in multiple sites in a synchronous learning environment where course material is presented and discussions occur for all students at the same time. It is reported that few studies have explored quality issues for programmes where students take a course completely on-line in an asynchronous learning environment and students have some control over when the material is viewed and when they wish to participate in discussions. They conclude that research on the teaching effectiveness of on-line courses is necessary for these courses to offer the quality that students expect. It is recognised that on-line moderators need to help students migrate quickly to the asynchronous environment to minimise learning disruption (Kiernan, Thomas and Woodroffe, 2001).
According to Mazoué (1999) the aim of effective online course design is to produce a network-learning environment where students are able to construct adequate models of a targeted knowledge domain and acquire competence in using them in real-life. Motivational support is provided to students, which is necessary for an effective online course. The question remains whether student perceptions of emoderation are markedly different from the identified literature.
The study is based on the on-line BA Enterprise programme, part of the Objective 1 funded ECW initiative designed by the University of Glamorgan, which aims to help improve the entrepreneurial capacity of Wales. The programme is funded by the European Social Fund (ESF) and supported by a network of Welsh Further Education colleges whose primary objective is to maintain and develop the learning in regional centres throughout Wales. The course is supported by “Blackboard” software and utilises a number of synchronous and asynchronous communication mechanisms including discussion boards, e-mail and virtual classrooms. Course materials are linked to “Blackboard” via a virtual learning environment (VLE) supporting text based learning material and case studies utilising graphical and audio techniques.
The methodology consisted of forty-minute interviews with students from the University of Glamorgan and partner colleges utilising a semi-structured questionnaire developed by the study authors. The aim of this research instrument was to elicit responses from respondents as to what factors constituted effective e-moderation and student perceptions of the role of the on-line tutor. The questionnaires asked the respondents to identify examples of effective and ineffective e-moderation practice, characteristics of the e-moderator and suggestions for “best practice”. Students from the University of Glamorgan and partner colleges were invited to take part in the study and thereafter a series of interviews were arranged from September to November 2003 in each of the centers. Thirty-five students from the 2nd and 3rd year of the BA Enterprise programme completed this process. The questionnaire results were then evaluated and cross-compared to identify commonality and key trends.
This section will appraise the results of the survey to identify what factors constitute effective emoderation and to rank these in order of significance. Thereafter ineffective emoderation is appraised and again ranked in order of importance. Examples of both practices are provided from reflection on emoderator experience. This section concludes by identifying the key characteristics of an emoderator.
|
Table 1: Survey Respondents
by Frequency & Percentage |
|||||
|
Age Groups |
20-30 |
31-40 |
41-50 |
51-60 |
61-70 |
|
Frequency |
3 |
11 |
12 |
9 |
0 |
|
% |
8 |
31 |
35 |
27 |
0 |
Thirty-five students from the BA Enterprise programme completed the interview process including 20 (58%) males and 15 (42%) females. The student population was mature with 62% aged over 41 (see Table 1). Only 8% were aged 20-30 years of age. The main explanation for the phenomena is the target market for the BA Enterprise programme. The purpose of the course was to encourage entrepreneurship within non-traditional learners. Students who undertook the course were typically middle aged and in employment and undertook to study the course around their working and personal lives.
Students were asked to consider what factors in their
opinion constituted effective e-moderation.
Five prime factors were identified namely regular and rapid feedback,
constructive and positive feedback, support and encouragement, enthusiasm and
organisation and direction. Each of
these prime factors is not considered in turn: - Regular and
Rapid feedback
Twenty-seven (77%) of students identified regular and rapid feedback as an essential element of effective e-moderation. Four respondents suggested a timeframe of 48 hours for e-moderator to reply to student communications. Students’ expected prompt and regular feedback from their e-moderators’ quotes included: -
“The better e-moderators respond regularly and quickly to
my needs”.
“I would like to see any email or discussion board thread responded to in 48 hours or less. I don’t think this is unrealistic and at the beginning of the course it should be less”.
Thirteen students (38%) identified the importance of constructive and positive feedback from e-moderators for coursework and weekly activities.
“Distance learning is different to more traditional
methods of study & requires student self regulation, so it is important for
e-moderators to foster & reinforce learning goals and targets, using
positive feedback”
“My time on line is precious. So what I need is realistic criticism of my work where both its strengths and weaknesses are identified and evaluated”.
Seven (19%) of students identified the importance of support and encouragement from the e-moderator. Students expected e-moderators to provide ongoing support and encouragement to individual students.
“Enthusiasm and support from the e-moderator is essential
to create a positive learning atmosphere”
Fifteen students (42%) identified the importance of e-moderator enthusiasm. This factor has obvious links with the support and encouragement factor.
“Enthusiasm in all e-moderator activities such as
responding to threads, enthusing about the subject matter and boosting your
confidence are all very important to me”.
Thirty-one percent (11%) of surveyed students
identified the importance of an organised e-moderator. Students liked e-moderators to demonstrate
several competencies including subject knowledgeability, organisation and
guidance and advice skills.
E-moderators that possessed these skills made the student learning
experience more enjoyable and structured and therefore less stressful.
“I find it important that the e-moderator provides
direction in terms of managing the learning in terms of weekly tasks. This includes directing, managing and
closing the discussion. Providing
summaries of these discussions is also useful.
Students were asked to rank the factors that constituted effective e-moderation in order of significance. Of these the top ranked factor was regular and rapid feedback with 19 first rankings. Thereafter constructive and positive feedback and support and encouragement were equally 2nd ranked with six top rankings. The significance of these factors is reinforced by average rankings whereby regular and rapid feedback attains the lowest average (1.38) followed by constructive and positive feedback (1.8) and organised and directed (1.86). E-moderator enthusiasm was considered of less importance with only one top ranking and an average of 2.62.
Students were asked to provide examples of effective e-moderation and how they had impacted on the student and their studies: -
“The feedback from the first assignments helped spur me
to work harder for the second. Thus at least I considered I was being rewarded
for the work done. The feedback was
both positive and also identified improvements”.
“The feedback so far has been positive and made me want
to contribute more. Leaving threads can
be a demanding experience because you want to leave something of value, which
often requires background reading. The
feedback has made me comfortable with this process and the value of my
contributions”.
“In the early stages of this course a helpful,
knowledgeable e-moderator makes the difference between finishing and staying on
the course”.
“I have personally had the need for a lot of support due
to personal problems since the start of the course. The tutors helped me through and encouraged me to carry on”.
“The feedback on the discussion board encouraged me to
contribute further and to research more into the subject that I was
studying. It also provided me with a
different perspective which broadened my view on the topic being studied”.
“I also felt encouraged when tutors replied to my
contributions, as I knew they had been read and my point of view has been
considered. As this was our main form
of communication, this contact was important”.
Sixty two percent (22) of survey respondents identified infrequent and delayed communication or feedback as a prime cause of student dissatisfaction. This factor included delayed, late or infrequent responses to student email or discussion threads and posting one general message to respond to several student messages. Student responses included: -
“Lack of response, which indicates forum postings are not
being read, makes the student feel isolated”.
“Lack of feedback and communication initially made me
feel isolated and I had to find my own solutions to the problems I faced. These were not always the best solutions and
sometimes caused me more work. This
resulted in great frustration and probably undermined my confidence”.
Nineteen percent (5 respondents) of students identified ineffective e-moderator organisation as a cause of frustration. These included the changing of dates and the reorganisation of course related issues at the last minute, the late return of assignment feedback and the lack of clear instruction in terms of task and assignment completion. Student responses included: -
“There was a lack of organisation from the e-moderator in
one particular module. We were very
much left to fend for ourselves with very little help or assistance. We had no idea when to complete tasks and
the e-moderator did not summarise the contributions, which had been made. I found the whole thing extremely frustrating”.
Negative or brief feedback was identified as an example of ineffective e-moderation by 58% (20 respondents). This behaviour included limited, brief and negative feedback to tasks and assignments, which undermined confidence and motivation. Student responses included: -
“Negative or brief feedback, both of these are
ineffective as they stunt a student’s confidence. It is paramount that behaviour is avoided”.
“Negative feedback can be very discouraging, and when
given impersonally with no encouragement or praise to ease the pain it can be
very damaging to progress”.
Survey respondents were asked to rank the identified factors of ineffective e-moderation. The problem of infrequent and delayed communications was identified as the most significant factor with 25 top rankings and the lowest average of 1.07. Thereafter negative/brief feedback (4 top rankings, 1.81average) and ineffective e-moderation/organisation (1 &1.96) were 2nd and 3rd ranked factors.
Students were asked to comment on the impact of ineffective e-moderation on their studies:-
“Online manner can have this effect. One e-moderator I have encountered did not
have a positive online personality. As
a result I avoided participating in this module. Their comments were often dismissive and did not encourage
further discussion”.
“I have not had feedback to all my submissions, but had
favourable responses to most so I cannot say that I was affected by infrequent
communication. However without communication
between the student and tutor there seems little point of contributing because
there is no way of knowing whether or not you are on the right track”.
“Feedback from assignments is varied. Some come back within days others take 4
weeks. This means checking email often
to see whether feedback has arrived.
Waiting for results can be nerve wracking. Feedback varies and in general I am not always sure where the
loss of points comes from. I find
feedback quite general”.
Students were asked to identify the key characteristics of the role of the e-moderator in 5 key words. Six main characteristics were identified (see Table 2) whereby the e-moderator’s ability to support and understand the student were given most credence with 92%, followed by organised and efficient 73% and flexible and approachable 65%.
|
Table 2: Characteristics of the Effective e-moderator |
||
|
Characteristic |
Frequency |
% |
|
Supporting, Understanding and Encouraging |
32 |
92 |
|
Organised Efficient and effective |
26 |
73 |
|
Flexible and Approachable |
23 |
65 |
|
Knowledgeability of Subject matter |
14 |
40 |
|
Motivated |
15 |
5 |
|
Facilitator and Communicator |
8 |
24 |
Our research has indicated that students have predetermined expectations of what they expect of an e-moderator and certain characteristics and behaviours are required to ensure effective e-moderation. The initial student perceptions of what to expect from the e-moderators is probably biased by previous educational experiences of traditional teaching. This phenomenon is enhanced within this particular study, as the respondents were non-traditional learners with the majority of the group over the age of forty. This study has identified a range of factors that students consider prevalent within effective and ineffective emoderation. Some factors can be considered essential skills including communication, feedback and organisation, whilst certain personal qualities and character traits were identified such as enthusiasm, support and encouragement, flexibility and approachability and knowledge. Effective e-moderation skills included regular and rapid feedback, constructive and positive feedback, support and encouragement, enthusiasm and organisation and direction. Ineffective e-moderation included infrequent and delayed communication and feedback, ineffective organisation direction and negative and brief feedback. As would be expected the ineffective e-moderator factors are the direct opposite of effective emoderation, although emoderator enthusiasm and support and encouragement were given less prevalence. This factor can be attributed to the fact that students were generally satisfied with e-moderator support and enthusiasm and considered them less significant than the other factors. When asked to identify the characteristics of the e-moderator the same trends emerged as the effective e-moderation query with the addition of factors such as knowlegeability and tutor’s flexibility and approachability. This research has reinforced the need for effective e-moderator skills and the evaluatory framework presented in Figure 1 is suggested as a mechanism through which to gauge the effectiveness of individual tutors by both tutors and students.
|
Figure 1: An Evaluatory Framework for Assessing E-moderation Competencies |
|||||||
|
Experience Scale |
Negative |
Neutral |
Positive |
Experience
Scale |
|||
|
E-moderator
Skills |
Infrequent
& delayed communication |
1 |
2 |
3 |
4 |
5 |
Frequent
& effective communication |
|
Negative
& brief feedback |
1 |
2 |
3 |
4 |
5 |
Constructive
& positive feedback |
|
|
Ineffective
Organisation |
1 |
2 |
3 |
4 |
5 |
Organised
and directed learning |
|
|
Personal
Qualities & Characteristics |
Lack
of emoderation enthusiasm |
1 |
2 |
3 |
4 |
5 |
Positive
Emoderation enthusiasm |
|
Ineffective
Support & Encouragement |
1 |
2 |
3 |
4 |
5 |
Effective
support & encouragement |
|
|
Insufficient
Tutor flexibility & Approachability |
1 |
2 |
3 |
4 |
5 |
Effective
Tutor flexibility &
Approachability |
|
|
Insufficient
Subject Knowledge |
1 |
2 |
3 |
4 |
5 |
Expert
Subject Knowledge |
|
This research has provided the course team with some interesting data with regard to what students expect in terms of practice and behaviour from the emoderating team. An open dialogue was encouraged to ensure that all e-moderating issues would be identified and addressed. Given that the BA Enterprise programme was a pilot e-learning course within the University of Glamorgan problems were bound to occur. For example traditional teaching staff within the University and partner colleges had to be trained in the use of Blackboard and thereafter develop their own online moderating style. Initially standards of practice were varied. However, a code of practice has emerged based mainly on the asynchronous communication mechanisms (discussion boards and email). Students can expect individual feedback within 48 hours from an email or discussion thread. In addition, emoderators are expected to open weekly activity schedules informing students and summarise completed discussions within forums. It is a key activity of the emoderator to guide discussion to ensure the learning activities have provided the students with sufficient knowledge to complete the assignment. Good practice is shared between e-moderators and standards monitored by a separate department.
In conclusion this study can be seen to update previous studies by academics such as Berge (1995) and Salmon (2000). It identifies that e-learning remains an emergent area in terms of pedagogy and learning environments. Therefore it must be noted that the earlier frameworks of e-moderation will be the subject of change as technological developments alter and enhance the role of the e-moderator. In addition the e-moderator framework will differ with each provider depending on the platform, technology and pedagogy. Our research indicates correlation with Salmon (2000) and Berge (1995) in terms of communication skills, content knowledge and personal characteristics. However, the ECW model places less emphasis on Technical capability as it has developed technical support mechanisms and there is more significance given to the tutor support role.
Alexander, S.
(2001) E-learning experiences, Education + Training, Volume 43, Number 4/5, pp.
240-248.
Anderson, T.,
& Garrison, D. (1999). ‘New Roles for Learners at a distance’, in C. Gibson
(ed.), Distance Learning in higher Education: Institutional responses for
quality outcomes, Madison, WI: Atwood Publishing.
Arbaugh, J.B.
(2001) How instructor immediacy behaviours affect student satisfaction and
learning in Web-based courses, Business Communication Quarterly, Volume 64,
Number 4, pp. 42-54.
Benfield, G.
(2000) Teaching on the Web – Exploring the Meanings of Silence, ultiBASE, http://ultibase.rmit.edu.au.
Berge, Z.
(1995). Facilitating Computer Conferencing: Recommendations from the
field. Educational Technology. 35(1),
22-30.
Berge, Z.
(1996) Characteristics of online teaching in post-secondary formal education,
eModerators, Berge Collins Associates.
Childs, M.
(2003) E-tutoring in synchronous and asynchronous environments, Interactions,
Centre for Academic Practice, University of Warwick.
Coppola, N.W.,
Hiltz, S.R. and Rotter, N.G. (2002) Becoming a Virtual Professor: Pedagogical
Roles and Asynchronous Learning Networks, Journal of Management Information
Systems, Spring, Volume 18, Number 4, pp. 169-189.
Cox, E.S.,
Clark, W.P., Heath, H. and Plumpton, B. (2000) Key Facilitation Skills for
Effective Online Discussion Groups: Herding Cats through Piccadilly Circus,
International Distance Education and Open Learning Conference, University of
South Australia.
Denard, H.
(2003) E-Tutoring and Transformations in Online Learning, Interactions, Centre
for Academic Practice, University of Warwick.
Drago, W.,
Peltier, J. and Sorensen, D. (2002) Course Content or the Instructor: Which is
More Important in On-line Teaching? Management Research News, Volume 25, Number
6/7, pp. 69-83.
Kiernan, M.,
Thomas, P. and Woodroffe, M. (2001) Does the medium dictate the message?
Cultivating e-communication in an asynchronous environment, Open University,
Walton Hall, Milton Keynes.
Mazoué, J.G.
(1999) The essentials of effective online instruction, Campus-Wide Information
Systems, Volume 16, Number 3, pp. 104-110.
O’Malley, J.
and McCraw, H. (1999) Students Perceptions of Distance Learning, Online
Learning and the Traditional Classroom, Online Journal of Distance Learning
Administration, Volume II, Number IV, Winter, pp. 1-11.
Pearl, A.
(2003) How does one guide the learner in online learning? 4th Annual LTSN-ICS
Conference, NUI Galway, pp. 185-189.
Picciano, A.G.
(2002) Beyond student perceptions: issues of interaction, presence, and
performance in an online course, JALN, Volume 6, Issue 1, pp. 21-40.
Salmon, G.
(2000) E-moderating: the key to teaching and learning online, Kogan Page,
London.
Salmon, G. and
Giles, K. (1997) Moderating Online, eModerators, Berge Collins Associates.
Volery, T. and
Lord, D. (2000) Critical success factors in online education, The International
Journal of Education Management, Volume 14/5, pp. 216-223.
Wang, Y-S.
(2003) Assessment of learner satisfaction with asynchronous electronic learning
systems, Information & Management, Volume 41, pp. 75-86.