University of Sheffield
This paper reports some of the findings of practice-based, case study evaluation research that examined participants’ experiences of a networked learning approach to professional development. The research revealed networked learners’ experiences of four broad, interconnected developmental processes - orientation, socialisation, communication and organization - and the impact of these on the nature and quality of engagement with the learning environment and tasks. The paper focuses on experiences of orientation, concluding by highlighting some practical evaluation implications arising out of this research, and by suggesting that it points to more general considerations for networked learning design and facilitation.
networked learning, case study, action research, evaluation
This paper highlights some findings from a detailed case study analysis of course participants’ experiences of a networked learning approach to professional development. Drawing on ideas from constructivist and situated learning theories, the seventeen-week course was conceived as a resource environment within which practitioners would carry out a number of flexible tasks that would enable them to explore ideas and develop skills of most relevance to their professional interests and circumstances. Emphasis was placed on developing new perspectives and expertise within a learning community; and a portfolio approach to recording learning was encouraged. Access was entirely on-line – there were no face-to-face meetings – and the technical platform was a ‘home grown’ experiment in virtual learning environment (VLE) design, in that the Web and a number of asynchronous and synchronous conferencing tools were used to provide integrated access to social and information resources. For all participants, this was a new type of learning experience. From the perspective of my role in developing the course design and as one of a number of course tutors, I embarked on this research with the aim of improving both my understanding of networked learning from the learner’s perspective, and the impact and effectiveness of my own educational practice (see Levy (2003) for a discussion of the action research methodology). The case study draws on a range of sources of data to (re)construct ‘what happened’ on the course and to explore the question ‘how should this be interpreted?’ in relation to (my own) educational objectives, assumptions and strategies. One key theme to emerge was the identification of four broad, interconnected developmental processes in experiences of networked learning, as follows:
· Orientation – becoming aware of, and positioned in relation to, key features of the learning environment, resources and approach;
· Communication – using computer-mediated communication as a means of self-expression and dialogue within the learning environment;
· Socialisation – developing interpersonal connections, relationships and community feeling within the learning environment;
· Organisation – planning, structuring, managing and directing personal and collective engagement with the networked learning environment, resources and tasks.
The research showed the impact of participants’ experiences in these four areas on the nature and quality of their engagement with the learning environment and designed tasks. Positive experiences contributed to positive engagement with the environment and tasks, whilst negative experiences placed constraints on productive engagement. The research also revealed a combination of factors that were perceived to shape learners’ experiences in these areas. Both enabling and constraining factors were identified, some of which are highlighted in relation to the process of orientation in this paper.
A somewhat different picture emerges here as compared with research that has portrayed the developmental process experienced by e-learners generically as a progression through clear-cut, sequential phases - and there would seem to be rather different implications for learning design and facilitation. Most influentially, perhaps, work by Salmon (2000) depicts e-learners’ experiences as a stage-by-stage progression from initial entry to the environment through a sequence of practices culminating in the adoption of constructivist approaches to e-learning. The design model stemming from her research recommends a stage-by-stage pedagogical approach to support progression from one set of practices to another, whereas the findings of this research suggest the value of a more integrated, holistic approach to ‘process support’ for networked learning.
Reflecting the findings of this research, Figure 1 depicts these four processes as central to productive, self-directed and collaborative networked learning within the context of the learning environment and design in question. It shows them as parallel, interconnected and mutually reinforcing processes, rather than as sequential phases experienced one after another. The developmental nature of participants’ experiences of these processes, in terms of improved personal awareness, relationships and practices as networked learners over time, is signalled on the diagram, as are variations in the ‘entry conditions’ for individual learners. However, the diagram does not show the considerable amount of variation in the speed and ease with which individuals moved through these processes. Nor does it show the potential for individuals to move at somewhat different rates through each one, despite their inter-connectedness.
Figure 1: Developmental processes in networked learning In what follows, the focus is on experiences of
(dis)orientation in the specific context of this course. Pseudonyms are used for individual
participants, who are also designated by numerical identifier
(P1-P33). Sources of data are
identified by the following conventions: OT – online transcript; QF –
questionnaire feedback; RC – research conversation.
Figure 1: Developmental processes in networked learning
In what follows, the focus is on experiences of (dis)orientation in the specific context of this course. Pseudonyms are used for individual participants, who are also designated by numerical identifier (P1-P33). Sources of data are identified by the following conventions: OT – online transcript; QF – questionnaire feedback; RC – research conversation.
Three main dimensions of orientation are identified:
· Orientation within the learning space: becoming aware of, and positioned within, the structure of the course Web site and the virtual spaces created by its CMC tools;
· Orientation to the learning design and approach: becoming aware of, and engaged with, the nature and practical implications of the learning design and approach. This involved two stages: firstly, engaging with information about the design and approach, and secondly, developing a deeper critical understanding of the implications of these for the practice of learning;
· Orientation within the information environment: becoming aware of, and positioned within, the electronic information resource environment within and beyond the course environment.
Orientation: “positioning with relation to specific directions; alignment of oneself or one’s ideas to surrounding circumstances” (Collins English Dictionary, 1999).
On entry to the course environment participants encountered the home page for the first, two-week, Unit. This presented them with a clickable image map of the Unit structure, timetable and its cycle of intended activity, illustrating graphically the relationship between individual, group and plenary tasks. Whereas some tasks were to be carried out in sequence, others were designed to run concurrently. The map was intended to show how participants should work through a cycle of activities in each Unit towards reflective ‘closing round’ discussion and individual portfolio work. They also encountered a number of asynchronous conferencing forums and a Web-based synchronous chat facility, as well as the course’s Resource Base - a structured, Web-based information resource comprising bibliographic references to off-line documents, annotated links to external Web documents and links to a small range of materials produced specifically for the course, including guidance materials about the learning approach that suggested approaches participants might take to tasks such as learning journals, project planning and portfolios. Unit 1 included some tasks that were intended to introduce participants to the learning environment and to the course objectives and learning approach, alongside other introductory tasks, including small-group tasks. I hoped that participants would read the guidance materials and raise concerns and questions in the plenary forum, thereby initiating general discussion of learning and support issues that might extend throughout the course.
Most participants, using the signposting provided by the Unit map, the Technical Support area of the Web site and the “Technical Issues” discussion forum hosted by the technical support tutor, found that they needed relatively little time during the early days of the course to gain a clear overview of the learning environment and to locate its key spaces and landmarks. Early technical concerns related to access to different areas were generally resolved quickly, and in common with the majority of others Esther later recalled that she had “picked up finding my way around the course fairly easily really” (P20:RC). Participants rapidly located the pre-established bulletin board forums - although not everyone visited all of these straight away – and as time went on found it easy to follow the pathway from one Unit to another. At the same time, it took a few participants somewhat longer than others to become fully aware of, and confident about, the structure of the site and the resources and facilities at their disposal. Angela later explained: “Looking back it is a clear structure, but I’m always a bit like that with Web sites, I’m always, have I seen everything? Have I been to all the bits? So I was disorientated at first” (P18:RC). Contrary to expectation, not all participants explored the learning space fully at the start of the course. Looking back, Claire explained that, “There were sections I didn’t discover until quite late in the course, like resources to do with different technologies, but maybe that was a time thing, I hadn’t had time to fully explore the site” (P10:RC). And as we shall see, the clear structure of the learning space did not necessarily lead straightforwardly to unproblematic engagement with the information environment or the learning design.
As they began to explore the extensive Resource Base within the course Web site, with its signposts leading out of the designed resource environment towards a rich landscape of digital resources, participants were becoming aware of, and beginning to engage with, the information environment within which the course was located. On the one hand, this richness was a source of excitement, and Naomi’s enthusiasm was shared by some from the outset: “So many good information sources, and they all led to others […] I really enjoyed going on that Resource Base and going from one link to another” (P24:RC). On the other hand, the extensive scope of this information environment often contributed to a sense of information overload. As Claire later said: “The team obviously put lots of effort into the Resource Base and it’s awful to say that didn’t work but when your starting point is very little knowledge, it feels as if there’s so much stuff you can’t even start” (P10:RC).
Despite their familiarity with the Web as information professionals, the borderlessness of the environment, combined with the seductions of hypertext, could be disorienting to them as learners. As Lydia put it, “One of the things I had not appreciated was that with an on-line course you never get to the end! When you are reading books or articles you do eventually get to the end but with links it’s never-ending!” (P27:QF). Similarly, Kate commented that, “It was hard, being prepared for the amount of material on the Web, it still seemed to go on forever even with the structured Resource Base you had […] I should have realised, I’ve had the Web at home and at work for 4 years, but I felt, where do I stop?” (P6:RC). And whilst one view was that the course’s information environment was no different to a library, another view was that ease of access in the on-line environment made selective engagement more difficult: “I tended to think they all looked interesting and I started reading everything, and then as the course went on you realise you’ve got to be selective, but I found it difficult to be selective when it was all on screen and all available” (Valerie, P28:RC).
Over time, as they became increasingly acclimatised to this environment, and focused in their learning on the course, disorientation and anxiety in relation to information tended to decrease and participants found it easier to become selective in their information seeking. For some this happened about half way through the course; for others, it took longer, as in Charlotte’s case: “I found it very difficult to itemise what I wanted to read and found I was gathering. Penny described how easy it could be, when surfing the Web, to become distracted from the purposes of Unit tasks, and how much effort was involved in remaining focused on specific learning purposes: “Every so often you had to say stop, and come back. Ask what is it I want to do here because of the limited amount of time, so I had to think what I wanted to do” (P14:RC). Some participants continued to spend a good deal of time exploring the Web, sometimes losing their way and becoming distracted from the purposes and focus of Unit tasks. Difficulties tended to remain as long as participants lacked clear learning goals that could provide a context for selecting relevant resources. Later, Margaret correctly surmised that: “I wouldn’t mind betting that the ones who were happiest with [the Resource Base] were the ones with lots of time or knew what they were looking for, and could be more selective immediately” (RC).
Some participants found it relatively unproblematic to rapidly engage with, and grasp the practical implications of, information and guidance about the course’s learning design - the signposts provided by supporting documentation, Unit Overviews and early bulletin board discussion with tutors being sufficient to enable them to gain a clear overview of the design and its underpinning philosophy, and to understand the nature and practical implications of the individual, group and plenary tasks that they were invited to carry out. For Richard, this all seemed “very explicit” (P7:RC) from the start. Frances later recalled that, “There were lots of guideposts I suppose, that’s what I’d say, [the course] was well guided to make sure you didn’t get totally left behind or go off the track” (P17:RC). However, for many the process of orientation to the learning design proved to be less straightforward. It was common to experience a period of disorientation at the start of the course, during which the learning design was perceived as indistinct and confusing.
Early in the second week, a plenary discussion thread was instigated to invite questions and discussion about the way in which the course was designed, and about particular tasks that might be unfamiliar, such as the learning journal and portfolio. I hoped that participants would read the guidance materials and raise any concerns and questions about practical implications in the plenary forum, thereby initiating critical and reflexive discussion of learning and support issues that might extend throughout the course. This elicited little feedback, despite the lively exchanges that were occurring at the same time in other areas of the conferencing environment, particularly the ice-breaker thread and the “Technical Issues” forum. At the same time, it was evident that discussions in most learning set (small group) forums were not taking off as intended, despite the efforts of tutors to set a discussion in motion there. In my research journal I noted the constraints on our control over participants’ activity in the online classroom, as compared with face-to-face settings, and also my ignorance about the way in which information provided about designed tasks was actually translating into activity. I felt uncomfortably in the dark about participants’ responses to the course approach, including whether or not they understood and were carrying out the sequence of tasks as designed. With the following posting to the General Issues forum I expressed something of my concern: “[…] Maybe everyone feels perfectly clear about how it all fits together and is just busy getting on with it, which is great! But please do feel free to ask questions, make comments […] we’re open to discussing any aspect of the course at all” (PL:OT).
Again, this invitation elicited some, but relatively little, response. Yet at this point many participants were, as Valerie put it, “struggling to find a conceptual map to cope with the course,” adding that, “I usually rely very heavily on face-to-face contact” (P28:OT). Simon also missed the face-to-face element: “I definitely needed the face-to-face. I found it so difficult to orientate myself” (P23:RC). Some participants found it difficult to identify what they could recognise as concrete tasks during the early weeks of the course; others were unsure of what the connections were between tasks, and which activities they should prioritise. Esther found it “very difficult to realise at first what I was supposed to be doing”, welcoming the MOO training session in Unit 3 as “something concrete to prepare in a straightforward way […] definite tasks to do” (P20:RC). Similarly, Ruth’s first few weeks were “very bewildering, you didn’t know what to concentrate on and what to spend more time on” (P4: RC). Margaret later recalled that, “I didn’t know where I was” (P21:RC) and Charlotte that she had been, “all over the place” (P13:RC). In this situation, some participants prioritised reading, finding that this offered a familiar point of departure at a time when they were encountering what was, on a variety of levels, a new learning approach.
Whilst early signposts and support intended to facilitate orientation to the learning design and approach met the needs of some participants, it was not unusual for others to fail to notice, or fully take in, early information, guidance and discussion about learning issues. Peter later suggested that this might be an inherent feature of the online learning environment, in that, “people flash in and out so quick… you have to work at concentrating on a course environment like this” (P12:RC). Margaret recalled that: “When I looked back at the early messages that you had sent in the first two weeks, and it was like, well I know I read them but I obviously didn’t take them in, and if I’d paid attention to what the messages were saying I’d have been a lot better off […] the instructions were there but it was like I hadn’t taken them in” (P21:RC). Getting used to the technical features of the environment could displace attention from other dimensions of the introductory Unit; as Teresa put it: “[the technology] took your mind off other facets” (P15:RC). Others found that the intensive activity in some plenary forums and the amount of information accessible via the Web site had a similar effect. Whilst reading could help to become familiar with the scope of the course and to identify a personal focus, it also had the effect of obscuring from view other, parallel, learning tasks: “I didn’t really register that there were activities that needed to be actively done, and I don’t really know why I didn’t. I think just because it’s a new way of learning you think in the old style, you do the reading” (Esther, P20:RC).
As a consequence, early pointers to, and guidance about, journals and portfolios often went unnoticed in the early weeks, with the result that some participants ultimately decided against embarking on them. Thus, Kate later commented that: “I must have missed that bit […] Later on in [the plenary] there were a couple of references to things we were advised to do right from the beginning, and I hadn’t appreciated [the journal] or taken it up” (P6:RC). Attention was also distracted away from intended activity in, and guidance about, learning sets. Claire later recalled: “When I went back and looked and there was something in Unit 1 that we should have got together and talked about, we just didn’t do that. I must have missed that completely […] I didn’t remember reading that we should get together and do that task!” (P10:RC).
And when pointers and discussions were not unnoticed, they were not necessarily sufficiently engaging. Early postings from tutors within learning set forums aimed to introduce and invite discussion about small groups within the learning design as well as the intended roles and contributions of set tutors and other members. However, relatively little discussion ensued at this stage and the role of sets in the learning design remained unclear (as well as unconvincing) to some participants for some time. Siobhan later commented that, “early on we were taking in a lot of stuff and it was explained about the learning sets, so I’d read that message and just not assimilated it” (P11:RC). Reflecting later on the slow start to her set, she felt that, “had we woken up earlier, clicked to the role of the learning set, then we would have worked [better]” (P11:RC).
At the same time, some participants developed misconceptions early on about the role of reading in the learning design, the importance of which, in relation to other tasks, tended to be over-estimated. Some participants approached the items in the Resource Base - beyond those that were marked as ‘essential’ - as an indicative list, seeing its wide scope, as we have already noted, as a positive feature. As Teresa later put it, “I didn’t feel oh God I’ve got to wade my way through this lot, I didn’t treat it like that” (P15:RC). Her perspective matched my own intentions and expectations. However, the perception that there was a very strong emphasis on reading on the course was common, especially at an early stage when Julia’s experience was widely shared: “To begin with I was quite worried about keeping up with it all and feeling I’d have to read everything that was recommended” (P16:RC). And as with other aspects of the learning design, not all participants were in a position readily to attend to, or assimilate, the guidance that was offered about this issue: “[Later on] I was trying to read everything and several people said, well you shouldn’t have done that, you should have been picking up what was relevant and of interest and not trying to do it all. But [early on] I wasn’t sure what was expected of me, even though I’m sure you said so, I hadn’t picked it up and I was trying to do everything” (Emma: P5:RC).
More broadly, participants often were not in a position to assimilate information rapidly about a learning approach that, with its emphasis on self-directedness, flexibility and reflection, was both unfamiliar and unexpected. As Jonathon later explained: “I didn’t pick up what the model was at the start, didn’t pay enough attention to it [my] expectation was that it was a course and you’d have the content dumped in to you […] in the early weeks I was still expecting delivery of course content coming my way.” (P29:RC).
With the failure of the plenary thread to encourage questions and discussion on learning issues at an early stage, and as individual participants expressed confusions and raised questions on a one-to-one basis with tutors or in learning sets, I came a better understanding of the importance of monitoring individual participants’ awareness of features of the learning design, and of the developmental nature of orientation to the learning approach. I and other tutors made efforts to clarify the design and explore learning issues with participants in an integrated way, alongside learning activities as the course progressed. I became more aware of the value of the small-group forums, as opposed to the plenary forum, in this respect, and some adjustments were made to supporting information on the Web site. Thus, the process of orientation to the design and approach took place over time, facilitated not only through early ‘induction’ but also through experiential engagement with learning tasks and resources, and opportunities for iterative ‘process’ dialogue with both tutors and peers. Julia later described how: “I think over the time you sort of developed the impression it was for you to decide which way you wanted the course to go, which I think was the aim really […] because it was really a learning experience aimed at you rather than an examined course” (P16:RC). As participants became more experientially oriented to the learning approach, initial perceptions of the learning design could change. A design that was at first perceived as indistinct later emerged as a clear and coherent whole; as Siobhan put it, “all completely integrated, there was a unity” (P11:RC).
At the same time, for many, orientation to the learning approach was still on-going as the course was coming to an end; participants were continuing to explore ideas about self-directedness and the implications for their own practice as learners. For example, the assumption that the onus was on participants to sort out difficulties, “rather than to run to teacher and say how do we do it” (P8:RC) was discussed, and it was not until late in the course that Helen felt she came to a clearer understanding the role tutors were aiming to play in support of self-directed learning: “I realised that was your job to support us, and you’d told us often enough, but until [then] I don’t think I really appreciated it” (P3:RC). Towards then end of the course, Jonathon remarked that he had only recently “started to see how the whole course was meant to hang together… all that stuff about reflective practice, constructivist knowledge and active learning is beginning to make (some kind of) sense!” (P29:OT). Looking back after the course had finished, he confirmed that, “It took me a long time into the course before I picked up on [the intended approach] It wasn’t until around about [Unit 5] that I went back and read a lot of the early stuff, and I thought blimey, that’s what we’re doing!” (P29:RC).
This case study account points to a range of issues related to the practice of networked learning design and facilitation including, perhaps, the limits of design in a networked environment (Jones, 2002). But it draws attention in particular to the question of supporting the process of orientation. From an evaluation perspective, the questions that arise are: how effective was this particular instance of networked learning design and implementation in this respect, and how might it have been improved?
In their post-course questionnaire feedback most participants evaluated the first Unit positively - many very positively - in terms of providing an introduction to the course and its environment. The design and usability of the course Web site and its computer-mediated communication tools were highly rated. As illustrated above, most found it relatively unproblematic at an early stage to gain a clear overview of the course’s learning environment and resources, and to navigate the space confidently. A rather less positive picture emerges in relation to orientation within the course’s information environment and, especially, in relation to learning design and approach. As the case study shows, participants often experienced a level of information anxiety and disorientation at the start of the course, and in engaging with information resources could become distracted from other aspects of the learning design. Re-orientation within the information landscape occurred over time, as participants explored what was ‘out there’ and came to judgments about the quality and relevance of the resources they encountered in relation to their learning purposes. At the same time, it is evident that participants did not always engage with early information, guidance and discussion about the various aspects of the learning design, and that coming to a deeper, critical understanding of the learning approach and its personal and practical implications was a developmental process for most, rather than a ‘once and for all’ event right at the start, and was facilitated by reflexive engagement in course activities and opportunities for on-going dialogue with peers and tutors.
Participants’ views on the effectiveness of support for orientation differed in the light of individual experiences; variation in this respect being affected by participants’ assumptions and expectations about learning. However, there was broad consensus that the approach to supporting orientation on the course, especially in the early weeks, had only partially been successful. The points that follow briefly highlight some (not all) of the practical evaluation points that emerged from the research in four broad areas.
· Task complexity was an issue at the start of the course. Being asked to carry out a number of tasks in parallel distracted attention from tasks intended to support learning orientation. The focus on learning issues could have been distinguished more clearly from other introductory activities, perhaps as part of a more extended induction period during which there also would have been plenty of time for exploring the learning space fully.
· Task specification at the start of the course in relation to consideration of learning issues might usefully have been more tightly structured (i.e., asking for more than general input to discussion), thereby sharpening the focus on learning orientation. At the same time, the emphasis on the use of the plenary conferencing forum meant that participants were expected to engage in a very public form of dialogue that many found daunting. The early focus on learning issues therefore might have taken place more effectively in small groups, led by learning set tutors, thereby setting the scene for further, on-going discussion about learning issues in sets. Reducing the emphasis on plenary discussion at an early stage in favour of small group activity also would have supported orientation (and perhaps encouraged commitment) to the collaborative learning aspect of the course more rapidly than was the case.
· On-going support for experiential learning, in the form of both informal opportunities and more structured tasks to encourage reflection and discussion about experiences of networked learning, proved especially effective in support of learning orientation, and could have been further strengthened.
· More use of synchronous computer-mediated communication tools early on in the course might have supported orientation more effectively, in terms of encouraging dialogue on orientation issues. A MOO environment (multi-user, object-oriented) was introduced later in the course with very positive effects in terms of communication and socialisation.
· Information overload was a problem in the early stages of the course. Exposure to fewer pre-identified resources initially might have been helpful as part of a step-by-step process towards orientation within the broad ‘information landscape’ of relevance to the course. Possible strategies might have been to defer the introduction to the full Resource Base until the second Unit, or to build up its scope incrementally as the course progressed.
· The case study confirms the importance of the online tutor’s role in relation to each of the dimensions of orientation. There was a need for more intensive and direct personal contact between tutors and participants in the early stages of the course, for example using email as well as the bulletin board, in particular to monitor and support individual participants’ awareness of specific features of the learning design and more general understanding of learning issues.
The issues that this paper has addressed are not unfamiliar in the context of pedagogies that aim to facilitate self-direction and collaboration in learning. It is well known that there are likely to be developmental implications for learners, including re-thinking of assumptions about the learning process, the self as a learner, and the roles of peers and tutors as well as the development of new skills; progression from disorientation to reorientation is recognised as part of this experience (e.g., Brookfield, 1986). At the same time, learning experiences are situated in particular contexts of social action. This paper has aimed to contribute to the growing body of knowledge about learners’ experiences in the networked environment (e.g., McConnell, 2000; Salmon, 2000; Tallman & Benson; 2000) by sharing something of my own learning from within that context, whilst also suggesting that the research may point to more general considerations for networked learning design and facilitation.
I would like to thank course participants for their involvement in this research, and colleagues (including course tutors and others) for their support during the process.
Brookfield, S. D. (1986). Understanding and facilitating adult learning. Milton Keynes: Open University Press.
Jones, C. (2002). Situation, learning and design: contexts for educational use of computer networks. Proceedings of ASCILITE 2002. [Online]. Available: http://www.unitec.co.nz/ascilite/proceedings/papers/101.pdf [5th August 2003].
Levy, P. (2003). A methodological framework for practice-based research in networked learning. Instructional Science, 31, 787-109.
McConnell, D. (2000). Implementing computer-supported co-operative Learning (2nd Ed.),London: Kogan Page.
Salmon, G. (2000). E-moderating: the key to teaching and learning online. London: Kogan Page.
Tallman, J. & Benson, A. (2000). Mental models and web-based learning: examining the change in personal learning models of graduate students enrolled in an online library media course. Journal of Education for Library and Information Science, 41(3), 207-223.