NLC2004 /Proceedings / Individual Research Papers
Chew Swee Cheng1,Chris Beaumont2, Seah Chong Poh1 and Gary Westhead3
Temasek Polytechnic IT School Singapore1, Liverpool Hope University College2, Edge Hill College of Higher Education3
chewsc@tp.edu.sg, beaumoc@hope.ac.uk, chongpoh@tp.edu.sg, westheag@edgehill.ac.uk
ABSTRACT
In the 21st century it is becoming increasingly common to work and learn in teams that are globally distributed. Such teams rely heavily on ICT to facilitate effective communication. There is a wide variety of communication tools and technologies to choose from and it is by no means obvious which tools should be used, and whether synergy exists when a communication environment is provided which includes a rich set of tools and media.
This paper reports and analyses the use of communication tools by students in a distributed Problem-based Learning (dPBL) environment. The PBL teams consisted of four students from Singapore and four from the UK. Teams had a rich set of synchronous and asynchronous communication tools available to them, including ISDN videoconference, WebCam Video/Audio, Microsoft NetMeeting® Chat, Asynchronous drop box and threaded discussion Forum.
This ethnographic study showed that semantic discussion threads included the use of all tools and enabled students to effectively co-construct meaning (and understanding). Students were able to deploy the use of different tools effectively to achieve the learning outcomes intended for various stages in the PBL cycle, with both asynchronous and synchronous media being considered of high importance.
Keywords
Problem-based Learning (PBL), Distributed Problem-based Learning (dPBL)
INTRODUCTION
In the 21st century it is becoming increasingly common to work and learn in teams that are globally distributed. Such teams rely heavily on ICT to facilitate effective communication. There are a wide variety of communication tools and technologies to choose from and it is by no means obvious which tools should be used, and whether synergy exists when a communication environment is provided which includes a rich set of tools and media.
This paper reports on a project sponsored by the British Council and Temasek Polytechnic to explore and analyze how students used ICT to support Problem-based Learning and construct shared understanding when teams are globally distributed.
PROBLEM-BASED LEARNING (PBL)
PBL is a teaching methodology in which an authentic, real-world problem drives the curriculum (Boud , 1997). Students work in small groups of about 4 or 5, solving problems presented to them and which are based on real work scenarios. Professional and funding bodies promote PBL as an appropriate strategy for professional education and it is increasingly becoming the method of choice (Newman, 2003). Ellis et al. (Ellis, 1998) suggest that “the computing discipline lends itself to PBL”. Distributed PBL (dPBL) is a variation of PBL that mirrors the situation of geographically dispersed work teams and therefore provided a very suitable basis for the study.
Successful Problem-based Learning requires effective communication for a variety of different purposes. These include social and team maintenance, problem solving, constructing understanding relating to the PBL case and task/process oriented related to organisational aspects. This communication is particularly challenging in distributed forms of PBL as Clark and Brennan, cited in Preece (1994, p.156) put it, participants, “….have to coordinate on process, they need to update their common ground moment by moment. All collective actions are built on common ground and its accumulation”.
The study in this paper employs a wide array of pedagogic and technological strategies in order to facilitate the accumulation of shared understanding and common ground needed, and to achieve the desired learning outcomes. These comprised both synchronous and asynchronous tools. Synchronous tools included ISDN video conferencing, WebCam video conferencing and synchronous chat (Microsoft NetMeeting/ Messenger). A specialized portal (VLE) provided asynchronous tools. The portal was developed at the Temasek School of IT, Singapore specifically to support PBL. Portal features used in this project were the asynchronous threaded discussion Forum, Drop-box, Peer and Self-assessment tools.
Each PBL team consisted of 4 undergraduate Information Systems students in the UK and 4 polytechnic students in Singapore. The sub-teams of 4 UK and Singapore students held local face to face meetings in addition to the use of the other communication media.
The PBL case consisted of a computer network security scenario, involving both theory and practical work. Students were required to identify risks and threats for the scenario and design a secure infrastructure. Both UK and Singapore sides of each team also had to construct logically identical demonstration networks using 5 PCs and associated network software / hardware. The PBL case lasted 6 weeks (including one week for preparation and presentation).
The PBL model used was pitched between one extreme of no specific predefined learning outcomes (McConnell 2002, p1) and the other extreme with clearly defined goals at each stage (Steinkuehler et al 2002, p26). This model was adopted taking into consideration the learner characteristics, faculty expectations and institutional requirements (Chew, 2000).
All students had prior experience of PBL and most had prior experience of both synchronous chat and asynchronous communication facilities through VLEs (e.g. WebCT). However, considerable attention was still given to the preparation of students (Woods, 1995), and Salmon’s (2000) 5-step model also informed the module design.
RESEARCH DESIGN
In this study, the central focus was on how technology, specifically the Portal, enabled participants in their collaborative work in a dPBL environment. We sought to understand how the participants mediate their learning through the use of the artifacts (that is, the technological tools) present in the environment. The research objectives were to understand:
- What influences the participants’ choice of communication tools in accomplishing the stages of the PBL cycle?
- How do participants use the different communication tools to achieve collaboration with other members of the PBL team?
The choice of tool was largely under the control of the students, the only constraints being relatively restricted access to ISDN Video conferences. Better understanding of these issues should enable us to construct more effective and integrated communications environments for learning.
The project ran in two action-research cycles. The first cycle (Sept-Oct 2002) provided an opportunity to collect and analyze data for students undertaking the PBL case. The results from this analysis and evaluation were used to inform the second cycle, where a further 16 students undertook a very similar PBL case in September-October 2003. In this second cycle the data analysis tools were refined and some lessons learned were applied to the communication tools.
A naturalistic inquiry approach was adopted in this study. We deemed this the most suitable approach since we were seeking to understand the process from the participant’s viewpoint (Koschmann 1996). The researchers were also tutor-participants in this dPBL environment. Both qualitative and quantitative data were collected and analysed. A combination of these techniques has been used in other studies (Miles and Huberman, 1994, p. 40-42 outlined various reasons for combining the quantitative and qualitative approaches put forth by numerous researchers). In this study, the quantitative data was collected by means of a questionnaire and this served as a preliminary stage to the analysis of the other, rather richer qualitative data. In fact there is something of a tradition of use of qualitative methods in the study of Computer Supported Collaborative Learning (CSCL) (Andreassen, 2000, McConnell, 2002 and Bjork 2002).
Qualitative data was collected in a variety of forms. The students saved synchronous chat logs and the portal provided a wealth of asynchronous postings, suitably time-stamped. Some ISDN videoconferences were recorded. Students also completed questionnaires and semi-structured interviews, which were video-recorded to enable us to understand their perceptions and motivations for particular actions.
RESULTS AND DISCUSSION
Use of Communication Media
Students were asked to rate the effectiveness of the available communication tools for each of the PBL activities (Table 1) using a (0-5) point scale. Team maintenance is an integral activity within PBL whereas the other activities constitute distinct stages.
Table 2. PBL activities
|
Clarification & Understanding Problem |
|
Identification and prioritization of Learning Issues |
|
Distribution of Learning Issues for research & Learning |
|
Learning & research |
|
Sharing of Learning |
|
Application of Learning – solving the problem |
|
Reflection |
|
Team maintenance / social |
|
Peer assessment |
The results are shown in Figures 1 & 2 and have been separated for clarity.
The y-axis is a simple summation of the responses, with a possible maximum of
75.
The students rated the Synchronous Chat, Portal Forum and Dropbox as being
highly effective for clarifying and understanding the problem statement and
identification and prioritization of learning issues with the Chat scoring
slightly higher than the Portal Forum and Dropbox. In terms of the distribution
of learning issues for research and learning by team members, students rated
the Portal Forum and Dropbox higher than the synchronous chat. They have found
that in these areas, the asynchronous tools offered them flexible timing in
that they had time to reflect on what had been posted before making their
contributions to the discussion.
Students soon rejected the WebCam video, as the quality was poor. Audio was also rejected in favour of chat, since differences in accent between the Singaporean and Liverpool-based UK students impeded understanding.
One important aspect to be pointed out is that the Singapore half and the British half of the PBL teams read the postings and chose to hold face-to-face meetings before posting their contributions, thus effectively reducing the online contributions to 2 sets instead of 8. This is important since one serious problem with asynchronous learning networks is that where the group number is more than [1]4, there is a tendency for information overload, sometimes the messages are even repetitive or irrelevant, causing a lot of frustration. This frustration was indeed reported by one team in the 2003-4 cohort, as they chose to conduct most synchronous chat as individuals from their home locations.
The results in Fig 2 show that in terms of Sharing of Learning and Application of learning, the asynchronous modes, namely the Portal Forum and Dropbox were considered most effective. The students used the Forum mainly to inform members about what they had deposited in the Dropbox, clarify issues raised by others or to give additional information about the postings. The Dropbox was designed in such a way that files deposited can be of any type and size. The work posted was only visible to members of the same team, a trusted environment that ensures the comfort of the contributors.
In terms of reflection, students rate ISDN VC (41), the Peer Assessment Instrument (36) and the Forum (35) as the media that had encouraged them to do self-reflection.
These results helped sharpen the focus of the research. At the next level of analysis, the question of how exactly did the participants use these media in their collaboration to accomplish their tasks will be examined.
Analysis of Discussion Threads
A grounded-theory approach was used for analysis, validation within the context of the project being provided by the four researchers. In the first phase of the project we were able to identify a number of semantic threads of dialogue that students developed which integrated a range of the communication media. These dialogues also displayed distinct milestones (McConnell 2002, p.74) and we have established some patterns of use, which lead to some preliminary recommendations.
Three distinct stages can be identified in students’ collaborative efforts, corresponding to the three stages of the PBL cycle.
Stage 1 Negotiating direction and Goals
As in any PBL programme, the participants began by negotiating the issues to be investigated. Here they discussed possible versions of the topology, challenged each other’s suggestions and clarified their own.
As students explored the learning issues they would like to investigate further, they kept each side updated. They also clarified preliminary issues about equipment given and technical terms used. At this stage, the Forum was used more frequently than the others were since it offered flexibility to the students in terms of taking time to ponder and reflect on an individual and group basis on what had been discussed so far. The Dropbox was used primarily to upload files so that the discussion had some focus for students. It is important to note that the 4 members on each side negotiated their own learning goals and issues and then posted their conclusions in the Dropbox. The discussion between the 2 groups was more focused in the sense that they only needed to refer to 2 possible versions.
Once the goals had been negotiated, it signaled the beginning of the next level of collaboration, the beginning of a period of research and gathering information. Whether the group was able to move on to this higher level of collaboration depended to a large extent on two factors:
- whether the members had established a sense of belonging to a learning community which depended on
- whether the goals and learning issues had been negotiated by all the members and that they were clear about what each had to do
Stage 2: Distribution of Work and Individual Research
At this stage, the use of media shifted from mainly the Forum to the Dropbox as the group members began conducting self-directed learning, discussing their findings with members from the same country in face to face meetings and posting them in the Dropbox for the other group to comment on.
There were also scheduled synchronous chats. The students used this tool primarily for decision-making and to seek agreement on issues that needed clarification. In one meeting they also used an electronic white board to clarify issues related to the setup of the network system. This took place after a good deal of independent research.
Stage 3: Preparation of Deliverables for Submission
At various points of the programme, the programme developers scheduled deliverables for the team to focus on. The team preparation for the submission of this deliverable began in stage two and is completed in stage 3. This occurred at the end of the discussion, after the two sides of the team had made decisions on the final version of the topology diagram during the synchronous chats.
Having a shared goal, in this case, accomplishing the final version of the network topology diagram provided the motivation for the team to move their collaborative process forward. It is important to have such “check points” for students so that both they and the tutors had a feel of the progress made by the team. The team in this study focused on the process of collaboration, evident from the way they went about seeking clarification and agreement from each other. The students’ reflective journal entries reinforced the researchers’ conclusions that the students had been able to use the communication tools effectively to develop trust, attend to the affective dimension of teamwork, and they could effectively negotiate consensus.
Analysis of the Programme Using Activity Theory
The Activity Theory has been used in the study of Human-Computer Interactions (Nardi, 1996) and more recently in the study of student collaborative patterns in telelearning scenarios (Andreassen, 2000) and distributed learning (Russell, 2002). In the current study, three activity systems can be discerned in the PBL programme, namely:
The Induction Workshop
The
main purpose of the induction workshop was to equip the students with the
skills to use the portal effectively and to collaborate in their teams in
problem-solving. These skills then became one of the mediating tools in the
second activity system.
The Initial Meeting
In
this system, the first contradiction appeared. The intended conferencing
facility was not set up in time due to technical problems and the students had
to resort to using the Microsoft netmeeting tools. Due to the poor video and
audio quality, compounded by the linguistic accents of the students, the
initial meeting was not as successful as the participants had hoped it would
be. However, instead of complaining, the students switched to chatting online
by typing out their messages, here the first instance of a conflict resolution
and problem-solving model was adopted by the students and successfully removed
the difficulty they experienced. In this system, the authors distinguished
between two outcomes, the intended outcome, which was to establish initial
contact between the students and the incidental outcome, which was the
establishment of a problem-solving model, initiated by the students
themselves.
The Programme
In this system there were several contradictions, namely, technical, group behaviour and assignment requirements and conditions. It should be pointed out that the students see these contradictions not as stumbling blocks to their collaboration. Instead they saw them as something they had to resolve as a group. In other words, the contradictions had become change agents that, through their resolutions had helped the group to gel.
CONCLUSIONS
In any ethnographic study the validity of the conclusions is heavily dependent on the context and we do not make any strong claims to be able to generalize these. However, we believe that our findings can help others in the design of collaborative environments.
The importance of the rich environment of communication tools became clear in this project, particularly given the global distribution of the teams. This rich set of tools enabled the students to overcome difficulties. For example students regarded synchronous discussion as essential for decision making. However, WebCam video and audio proved ineffective because of bandwidth, latency and language difficulties, though we may expect the technology to improve in the future (Knutsen, 2003). Rapanotti (2002) also reported benefits from the deployment of voice groupware in a small trial, despite reliability problems.
The students overcame these issues by using Microsoft NetMeeting® chat. They also demonstrated flexibility to arrange meetings to overcome the seven-hour time difference.
Asynchronous Dropbox and Forum facilities were widely used for complementary tasks, such as exchange of information and product delivery (Dropbox) and challenging contributions and constructing shared understanding (Forum).
Our analysis shows that the semantic threads incorporated all of these tools in a rational way, and this emphasizes the need for an integrated set of such tools to encourage the development of high-performing teams.
It is more difficult to draw conclusions regarding the ISDN Video conference. It provided good quality audio and video, but technical difficulties restricted its use to one conference at the end of the first phase. It was also the only tool that was not available ‘on-demand’. However, students found this useful for group reflection, though there was initially some reluctance to talk.
In the second phase (Sept-Oct 2003) three videoconferences were scheduled, at the start (for team orientation / introductions), middle and end (for group reflection) of the PBL case. The teams used the middle conference to resolve team conflict and organizational issues.
All the teams involved in this study achieved a high quality solution that incorporated significant input from both UK and Singapore parts of a team. On no occasion did any sub-team withdraw to produce their own solution. They successfully demonstrated interdependence and co-construction of knowledge. Students also achieved the learning outcomes at a high grade.
Cultural differences caused by international distribution of students were not investigated in any detail within this project, although it emerged as a factor in selecting communication tools.
ACKNOWLEDGMENTS
We would like to acknowledge the sponsorship and support of the British Council, Temasek Polytechnic and Edge Hill College of Education in making this project possible.
REFERENCES
Andreassen, E.
F. (2000). Evaluating how students organise their work in a collaborative
telelearning scenario: An activity theoretical perspective. Masters Thesis.
Accessed on 15 June, 2003. http://www.ifi.uib.no/docta/dissertations/andreassen/index.htm
Boud D, &
Feletti, G., Ed. (1997). The Challenge of Problem-based Learning, 2nd ed.
London: Kogan Page.
Björk, U.
(2002). Distributed Problem –based learning in social economy – key issues in
students’ mastery of a structured method for education. Distance Education, Vol
23, No.1.
Chew, S.C.
(2000). Preparing students and faculty for educational change. Paper presented
at the 2nd Asia Pacific Conference on Problem-based Learning,
Ellis, A. et
al. (1998). Resources, Tools, and Techniques for Problem Based Learning in
Computing. ITiCSE '98,3rd Annual Conference on Integrating Technology into
Computer Science Education, ACM/SIGCSE. p42.
Goodyear, P.
(2001) Effective networked learning in higher education: notes and guidelines.
Centre for Studies in Advanced Learning Technology, Lancaster University.
Accessed on 17 Oct 2 003 at http://csalt.lancs.ac.uk/jisc/guidelines_final.doc
Knutsen, D.,
Knutsen, E., & Slazinski, E. (2003) Employing new advances in IP
videoconferencing to enhance teaching and learning through the use of a hybrid
distance learning course. Proceedings
of the 4th conference on information technology curriculum on Information
technology education Lafayette, Indiana, USA . 72 – 75.
Koschmann, T.
(ed) (1996), CSCL: theory and practice of an emerging paradigm. Malwah, NJ:
Lawrence Erlbaum Assoc.
McConnell, D.
(2002). Action research and distributed problem-based learning in continuing
professional education. Distance Education, Vol. 23, No.1. p1.
Miles, M.B.
& Huberman, A. M. (1 994). Qualitative data analysis, 2nd ed. Thousand
Oaks, CA: SAGE Publications.
Nardi, B.A.
(ed) (1 996). Context and consciousness: activity theory and human-computer
interaction. Cambridge, MA: MIT Press.
Newman, M.
(2003). A pilot systematic review and meta-analysis on the effectiveness of
Problem Based Learning, Learning & Teaching Subject Network Centre for
Medicine, Dentistry and Veterinary Medicine (LTSN-01) UK. p5.
Preece, J. et
al (1994). Human-computer interaction. The Open University, Addison-Wesley Ltd.
Harlow, England p156.
Rapanotti, L.,
Blake, C. T., & Griffiths, R. (2002).
eTutorials
with voice groupware: real-time conferencing to support computing students at a
distance Proceedings of the 7th annual conference on Innovation and
technology in computer science education, ACM/ SIGCSE Press New York, NY,
USA p116-120.
Russell, D.R.
(2 002), Looking beyond the interface: activity theory and distributed
Learning’. In R.M Lea and K. Nicoll, Distributed learning: social and cultural
approaches to practice. London: RoutledgeFalmer.
Salmon, G.
(2000). E-Moderating, the key to Teaching and Learning online. London : Kogan
Page.
Steinkuehler,
C. A., Derry, S. J., Hmelo-Silver, C.E. & Demarcelle, M. (2002). Cracking
the Resource Nut With Distributed Problem-Based Learning in Secondary Teacher
Education. Distance Education, Vol. 23, Issue 1, p23-39.
Woods, D. R.
(1995). Problem-based learning: Resources to gain the most from PBL. Hamilton,
ON: Don Woods.