NLC2004 /Proceedings / Individual Research Papers
University of
Southampton
ABSTRACT
In this paper, the theoretical perspectives used in a study of on line learning are described. Using activity theory as an organising principle, a group of science students were brought together on line to produce a knowledge-building community. Using a methodology involving both content analysis and group interviews, the interactions between members of the community were studied, based on ideas drawn from the sociology and philosophy of science. Results from this study suggest that this methodology provides a powerful technique when exploring on line learning, producing deeper meanings from data than would be possible using content analysis alone.
Keywords
learning communities, activity theory, content analysis, epistemology
INTRODUCTION
The work described in this paper began in 1995 with the founding of ScI-Journal (www.sci-journal.org) as a place on the web which would enable young science students to engage in discussing reports of science investigations, with the intention of promoting a greater understanding of science concepts, and possibly of the processes through which scientific knowledge is produced too. Initially such discussion was undertaken through the use of e-mail, with authors’ addresses being given at the top of each report. Issues relating to child protection led to the introduction of discussion lists in 2000, each report having its own list which is situated at the end of the report. In the current iteration of ScI-Journal, discussion lists are generated “on the fly” whenever a page is requested, with the comment data residing in a tsv (tab separated variable) log file. As comments are added to a report, the log file grows, and provides a record of comments and related data (not all of which is published in the discussion list). Taking Henri’s definition of interactivity (Henri, 1995:150-2), analysis of these log files shows that the majority of messages in the discussion lists may be characterised as “not interactive”, on the basis that, although they relate to the subject of the report, there is no evidence to suggest that there is any (re)construction of knowledge involved. The purpose of this research was therefore to establish an on line environment in which interactivity related to practical science investigations could flourish, and then to study the accounts of such interactions in order to provide some understanding of their nature. The specific questions addressed were:
- What do the on line interactions reveal about student epistemologies of science as they engage in knowledge-building?
- When students evaluate knowledge claims made by other students during an investigation, on what do they base their judgements?
This paper is concerned with the background theories used to frame the learning community in the study, to establish the nature of the data to be collected and to develop the instruments used to analyse the data.
Theoretical framework
Reports of research in which computer supported learning environments are used to examine aspects of learning in an on line community are now well-established in the literature. Much of this research is framed by the idea that members of an electronically connected group constitute a community of practice engaged in situated learning (Lave & Wenger, 1991; Wenger, 1998), and uses the analysis of computer databases containing information about interactions within the community as a means of understanding the learning processes happening within that community. Whilst at first sight it may appear quite legitimate to take a similar stance to approach the study of a group of science students engaged conducting a practical science investigation, there is good reason why one should, in fact, choose not to do so.
Social practice
Students engaged in learning in a conventional (classroom) setting generally do so under the guidance of a teacher, tutor or lecturer; in an on line setting, they may work with someone called a tutor, moderator or expert. Such an individual is sometimes said to play a rôle as a master, whose job it is to conduct the learning environment in such a way that the learners develop an ability to perform a social practice through a process that Lave & Wenger term cognitive apprenticeship (Lave & Wenger, 1991:29). If this view is also to be adopted when considering students learning science, then two points must successfully be argued: first, that the setting in which such science learning takes place employs the same kinds of practices (Wenger, 1998:45-50) as those found in authentic scientific communities undertaking research; and second, that the individual leading this process is accomplished in such practice. In the case of school science classrooms, both claims are highly suspect; and it is unlikely that learning situations in undergraduate classes in Universities are able to satisfy the first requirement even when they are able to satisfy the second. It is therefore doubtful whether it is ever possible, at a level below research studentship, to say that learning science (or more properly, learning to do science) is in accordance with theories based on common practice. Hakkarainen (1998:8) similarly notes the differences in practice between school classrooms and ‘authentic scientific enquiry’.
Whilst it is problematic to frame research in which groups of students carry out practical science investigations using these kinds of models, the need to employ some kind of framework which addresses the social aspect of scientific enquiry remains. In addition, to be maximally useful such a framework also needs to address other aspects of the processes of learning through and about scientific enquiry, not least those which concern the influences which mediate these processes (such things as scientific theories and epistemological stances) and the rules whereby the processes are regulated. Of course, the model must also give due recognition to the fact that a group of students are unlikely to learn unless someone in the community acts as a “guide” to learning in some sense; another mediating influence. One possible approach to address these requirements is to employ activity theory.
Activity theory
Activity theory is a long-established branch of cultural-historical research
with its origins in the Russia of some 80 years ago. Its name is somewhat
confusing, since it is neither directly concerned with activity per se, nor is it a theory in the sense
of being a fixed body of accurately defined statements (Kuutti, 1996:25). At
the heart of activity theory lies an attempt to understand the ways in which
individuals cooperate in communities in order to perform some kind of activity
directed towards an object; the basic unit of study for such a process is known
as an activity unit (figure 1). The
object may be tangible (eg an aeroplane) or it may be completely abstract (eg a
theory); the one requirement is that it should be something that all members of
the community can share for the purposes of manipulation and transformation
during the activity. Working together, the community
will employ a variety of mediating influences in pursuit of their object, including physical tools and tools used for thinking, and in doing so will be subject to a
variety of rules which govern the
conduct of the community (including the relations between the subjects making up the community) and
the division of labour within it.
As is already apparent from the above description, although activity theory provides a framework which (potentially at least) offers a way of describing and analysing the complexity existing when a community undertakes an activity, any attempt to do this in a holistic way is likely to fail due to the large number of interdependent relationships that must be considered. One way around this is to consider triads taken from figure 1, an approach described by Lewis (1997). It was the use of triads which informed the approach to understanding collaborative work during practical science investigations described here.
Applying activity theory
In this study of a group of student scientists undertaking a practical science
investigation, three triads were used as a theoretical basis for the empirical
work in order to seek answers to the two research questions described at the
start of this paper. The triads were labelled SCO, RCO and TCO in relation to
the labels in figure 1 – the three triads themselves are shown in figures 2a-c.
Triad SCO has its importance in the way in which the study itself was set up,
whilst triads RCO and TCO guided thinking about the collection and analysis of
data.
Triad SCO
The first of the three triads (figure 2a) describes the relationship between subject, community and object. Lewis considers that this triad plays an important rôle in the establishment of a learning community because “there must be space for interpretation, negotiation and the establishment of group ownership of the motive [the object of collaboration]” (Lewis, 1997:214). Here this was also judged an important triad due to the rejection (in this specific context) of the notion of social practice described earlier. Bereiter and Scardamalia also reject the idea that education below a certain level embodies social practice, noting that “children enter school at the age of five or six, but it is usually not until they are close to 20 that they encounter any serious effort to develop them into experts” (Bereiter & Scardamalia, 1993:184). Yet Bereiter and Scardamalia also note the enormous success of research groups in finding answers to problems and in generating new knowledge (which they term knowledge building) and suggest that knowledge-building communities could be employed to help students develop enhanced understanding, based on the idea that, although students are not actually constructing new knowledge, they are constructing new meanings for themselves. Sutton (1992) too argues for a focus on the social construction of meaning rather than on the interpretation of phenomena in order to enhance the quality of science learning.
The community of student scientists on which this study is based was brought together to constitute a knowledge-building community which would be encouraged and managed in such a way so as to encourage the development of nine characteristics of such a community:
1. sustained study of topics in depth rather than superficial coverage;
2. focus is on problems rather than on categories of knowledge;
3. inquiry driven by students' questions;
4. explaining is the major challenge;
5. day-to-day focus is progress toward collective goals of understanding and judgement;
6. students typically work in small groups, each with a different task;
7. discourse is taken seriously;
8. teachers can contribute what they know to the discourse, but there are other sources of information;
9. teacher's role shifts to leading it by virtue of being a more expert learner
(adapted from Bereiter & Scardamalia, 1993:210-11).
Based on this, the object of this particular activity unit is the social construction of an agreed (scientific) way of understanding the particular phenomenon under investigation through a process involving peer discourse and review.
Triad RCO
This triad (figure 2b) is associated with the protocols used by the community in rejecting or accepting knowledge claims made within the community during knowledge-building. As such these protocols may be explicit or implicit.
Explicit protocols are those which are agreed and adhered to by members of the community as they go about their tasks, whether as part of the wider social arrangements that exist in the institution within which they work or in society as a whole (eg ‘treat other people’s ideas with respect’), or as part of the knowledge-building community. In the case of a community engaged in building science knowledge, such protocols can usually be traced back to Merton’s four norms of science (Merton, 1973), which ascribe to science (and to scientists) the characteristics of communalism (science as public knowledge), universalism (science as a collective activity), disinterestedness (science as emotionally neutral) and organised scepticism (science as involving rational procedures and reserved judgement); these are sometimes abbreviated in the acronym CUDOS.
Implicit protocols are those which the group may operate without ever making their basis clear; or perhaps even without any discussion. In the context of science knowledge-building, of particular interest are the non-technical reasons which may be given for the acceptance or rejection of knowledge claims from within the community. Through extensive interviews with practising scientists, Collins and Pinch have explored the non-technical criteria these individuals say that they use when deciding the validity of a particular piece of work, and have produced a list which includes: personal knowledge of a scientist’s experimental capabilities and honesty; whether or not a scientist worked in industry or academia; a scientist’s style and presentation of results; a scientist’s ‘psychological approach’ to experiment; the size and prestige of the scientist’s university of origin (Collins & Pinch, 1994:101). This list is the basis of an instrument used to examine whether student scientists employ similar implicit protocols in scrutinising the knowledge claims of others from within their community.
Triad TCO
The third triad (figure 2c) involves the tools used by the community in addressing the object of the activity – the joint construction of an explanation for the phenomenon under investigation. In common with triad RCO, the tools may be regarded as explicit or implicit.
The term explicit tools relates to those which are applied through general agreement within the community and which are derived very largely from sources within the community of professional scientists. Such tools include science knowledge or ‘facts’ (eg the atomic mass of sulphur or the density of water) and scientific theories that are regarded as ‘proven’ (eg Newton’s laws of motion or the laws relating to chemical combination of elements). These tools may be acquired by the student knowledge-building community from a variety of authoritative sources which include text books, the teacher and the Internet (with especial regard to provenance I the case of the last example).
Implicit tools are similar to implicit protocols, in that they are operated by the community with little or no discussion. Especially important in this context is the approach to scientific knowledge and the relationship between evidence and theory used by the community – its epistemological stance. Whilst there is no agreement amongst scientists, philosophers of science and sociologists of science about what comprises a ‘correct’ understanding of the nature of science, there is some agreement about what may constitute an understanding that can be characterised as ‘naïve’. Nadeau and Desautels have described an epistemological stance that they call naïve scientism (Nadeau & Desautels, 1984). This has five constituents: naïve realism, blissful empiricism, credulous experimentalism, blind idealism and excessive rationalism; it forms the basis of an instrument used to investigate the epistemological stance of student scientists in the knowledge-building community.
empirical study
Whilst the stimulus for this work was the original work carried out with ScI-Journal, there are compelling reasons for the use of computers to support knowledge-building. In establishing and maintaining a knowledge-building community of students, networked computers can play a crucial rôle in at least three respects:
1. they can provide access to records of what other students are doing, and an opportunity to comment upon it;
2. they can provide access to other sources of information (in addition to the teacher and books);
3. they can provide access to experts outside the community.
If computers are used as a means of storing students’ thoughts and ideas, and as a way of recording the community’s communications, then students have access to a comprehensive, searchable source of information about what others in the community are thinking, and to a record of the development of the community’s progress towards their goal. This is very close to the way in which knowledge-building in an expert community is carried on; as Bereiter and Scardamalia point out “the community database serves as an objectification of the group's advancing knowledge, much like the accumulating issues of a scholarly journal” (Bereiter & Scardamalia, 1993:212).
Pilot study
Before moving to a full-scale study of a knowledge-building community based on the work described above, a small-scale study was conducted with a group of students in order to ascertain the technical feasibility of such work. Specifically, the questions asked were:
· is it technically feasible to support a science investigation using an on line discussion environment?
· will students use it?
· if so, will there be much (any) interaction of any value to research?
To obtain answers to the questions, a group of students (N = 21) with ages in the range 13 to 14 years in a comprehensive school were provided with continuous access over a ten day period to an asynchronous web-based discussion environment; this was in association with a practical investigation into friction that they were undertaking as part of their science work. The students were all in a “top” ability group (as defined by National Curriculum tests, nationally applied tests in such areas as cognitive reasoning, and by school science tests). The researcher and teacher acted as “experts and animateurs” (Henri, 1995) for the on line discussions. Students were introduced to the discussion environment in a short session in school at the start of the ten day period; this session had a purely social focus, and there was little or no discussion of science expected. Students were encouraged to make use of the environment both in and out of school.
At the end of the ten days, log files of the on line discussions were subjected to content analysis using a modification of Henri’s model (Henri, 1991), in which contributions were classified by the researcher into three broad categories:
· social – unrelated to the science topic under discussion
· technical – concerned with use of the on line environment
· cognitive – related to the science topic under discussion
The total number of contributions to the discussions during this period was 350, of which 101 were posted outside school hours. Analysis of the classification of out of school contributions showed that 51% were social, 8% were technical, and 40% were cognitive (Σ ≠ 100% due to rounding). Average length for social contributions was 9.3 words, for technical contributions 43.9 words (the majority of these by researcher and teacher in response to student difficulties) and for cognitive contributions 30.9 words. These figures were taken as encouraging, on the basis that a significant proportion of responses were related to the topic under investigation, and the length of these contributions suggested that students were engaged with the use of the on line discussion forum in a positive way.
Following the analysis, a group interview was conducted with a sample of students to ascertain their general feelings about the use of the on line discussion forum to support their science work, and to provide guidance about the details forum design for the main study. The teacher was also interviewed to ascertain his opinions as to the quality of the on line discussions (based on involvement in the discussion and on inspection of the log files) . Both teacher and students expressed their enthusiasm for the on line environment, and provided useful comments about its further development, leading to the conclusion that the answers to the questions posed at the start of the pilot phase were all in the affirmative.
Main study
The main study was split into two phases. The first phase involved two parallel classes from the same school as the pilot study, one of these classes including students from the pilot. This study was conducted some five months after the pilot study, in order not to interfere with preparation for National Curriculum tests. The second phase involved three parallel classes of students in the age range 13 to 14 years, taken from two schools. Although one of these schools was the school involved in the first phase and the pilot, all of the students in phase 2 were new to the research.
These phases of the study made use of a phpbb asynchronous discussion forum (www.phpbb.org) available continuously during each phase of the study, a synchronous chat room available at certain times (agreed between researcher, teacher and students) and a web-based area for file upload and download, to enable students to share such things as spreadsheets, digital photographs, word processor files and other electronic resources.
Data collection
Content analysis as applied to CMC in a formal educational setting relies on the process of categorisation to provide insight into the different kinds of contributions made by students as they engage in on line discussion. For such a process to be reliable implies not only consistency across time in the coding of contributions by a single researcher (internal reliability) but also where there are two or more researchers consistency between researchers (inter-rater reliability). However, even where studies can be shown to be reliable (and a review of 19 major studies using content analysis (Rourke et al., 2001) found that none could claim this), there must be concern over the validity of such studies, in the sense that the analysis is based on an imposition of the researcher’s meaning on the text in the log file; text which has be written by a student, who could quite legitimately argue that this text is imbued with a completely different meaning. The need for an approach to the analysis of CMC-derived data which incorporates some form of triangulation is a theme also developed by McConnell (McConnell, 2000).
Given these arguments, the approach adopted when collecting data in the research was one in which initial interpretation of CMC-derived data used content analysis to categorise and sort contributions to on line discussions. This made possible further data collection through group interviews which provided participants with an opportunity to contribute to the construction of meaning from the original log files. The practical choice of a school setting in which to carry out this work made this a feasible proposition, and meant that there was also an opportunity for teachers to contribute to the process of meaning making, bringing their knowledge of the students to bear on the analysis.
Data analysis
Log files from both synchronous and asynchronous discussion forums were subjected to initial content analysis, making use of a second modification of Henri’s model (Henri, 1991), in which contributions were categorised as follows:
· social – unrelated to the science topic under discussion
· technical – concerned with use of the on line environment
· cognitive – related to the science topic under discussion
· metacognitive – concerned with the nature of knowing
Selected metacognitive statements (which were always found closely associated with cognitive statements) were the subject of discussion with the students involved in the context of a group interview, in which the researcher provided the student(s) with the opportunity to expand on the on line contributions that they had made.
The implicit protocols (triad RCO) and tools (triad TCO) were explored by analysing log files and records of student interviews using instruments derived from Collins and Pinch (Collins & Pinch, 1994) in the case of implicit protocols, and from Nadeau and Desautels (Nadeau & Desautels, 1984) in the case of implicit tools. Clarification of the meaning of on line data, coupled with the use of data from the group interviews, enabled the clarification of meaning with the original authors of on line logs, together with an opportunity to explore issues in more depth. In many cases the on line logs provided a stimulus for discussion that took conversations far further than the original text-based electronic communications appeared to indicate. One such example involved students who were discussing the use of digital cameras and image manipulation software in making accurate measurements of a fast-moving object (table 1).
|
Karen |
In Valerie’s experiment we just sprung it and then measured how high it went … but in Martin’s one he could, like, replay it, so where we could just see it at that moment so we’d have to make sure it was there, Martin could, like, replay it and make sure it was there. So more like specific … make it to the millimetre if he wanted to. |
|
Linda |
It’s more reliable. |
|
Int |
Oh, it’s more reliable [emphasis]? Now what does that mean, Linda? |
|
Linda |
Well, it’s like – I don’t know … Valerie and Karen said they were marking it by the eye, and like the camera can like never lie, so … |
|
Int |
Right, OK, so because they used a camera you think that’s going to make it … |
|
Linda |
… it’s gonna like be more accurate, accurate. |
|
Int |
It’s got to be more accurate because they used a camera? |
|
Linda |
Well, yeah! |
|
Int |
OK, that’s fine. One of the things that someone’s written about the camera … Karen said “how were you [Martin] sure that you read the number correctly”. |
|
Karen |
Oh no, I was thinking differently, I thought that they were plugging it into the computer and reading it off the screen … |
|
Int |
Oh, right … |
|
Karen |
…cos if you go into the computer it gets like blurrier as it gets bigger. |
|
Table 1 |
|
Whilst initial indications from the on line logs were that the students involved were exhibiting a quite naïve view of the nature of knowledge, clarification in this subsequent interview showed quite the reverse. Karen and Linda’s discussion in the interview showed that Karen in particular appreciated the significant differences in the nature of evidence produced by enlarging an image using a digital camera’s optical zoom and enlarging it using an image-manipulating program on a computer; an understanding of the relationship between evidence and theory that was really quite impressive for this age.
conclusions
The use of activity theory as an overarching perspective to inform the structure of this study together with the associated data collection and analysis shows that this approach has much to commend it, not least in the way in which it can focus attention on pertinent aspects of the complex interactions within a community engaged in an activity aimed at a common object. In addition, the combination of content analysis of on line logs followed by group interviews to expand on initial interpretations can provide a rich and extensive source of data, with meaning constructed by researcher working in partnership with researched.
references
Bereiter, C.
& Scardamalia, M. (1993) Surpassing ourselves: an inquiry into the nature
and implications of expertise. Open Court, Chicago.
Cole, M. &
Engeström, Y. (1993) A cultural-historical approach to distributed cognition,
in Distributed cognitions. Salomon, G. (ed.) Cambridge University Press,
Cambridge, 1-46.
Collins, H. M.
& Pinch, T. J. (1994) The Golem: What Everyone should know about Science.
Canto edn, Cambridge University Press, Cambridge.
Hakkarainen,
K. (1998) Epistemology of Scientific Enquiry and Computer Supported
Collaborative Learning. Unpublished PhD thesis, University of Toronto.
Henri, F. (1991)
Computer conferencing and content analysis, in Collaborative Learning Through
Computer Conferencing. Kaye, A. R. (ed.) Springer-Verlag, Berlin, 117-136.
Henri, F.
(1995) Distance Learning and Computer-Mediated Communication: Interactive,
Quasi-Interactive or Monologue? in Computer Supported Collaborative Learning.
O'Malley, C. (ed.) Springer-Verlag, Berlin, 145-161.
Kuutti, K.
(1996) A Framework for HCI Research, in Context and Consciousness: Activity
Theory and Human-Computer Interaction. Nardi, B.A. (ed.) MIT Press, Cambridge
MA, 17-44.
Lave, J. &
Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation.
Cambridge University Press, Cambridge.
Lewis, R.
(1997) An Activity Theory to explore distributed communities. Journal of Computer Assisted Learning, 13, 210-218.
McConnell, D.
(2000) Implementing Computer Supported Cooperative Learning. Second edition
edn, Kogan Page, London.
Merton, R. K.
(1973) The Normative Structure of Science, in The Sociology of Science, Storer,
N.W. (ed.) University of Chicago Press, Chicago.
Nadeau, R.
& Desautels, J. (1984) Epistemology and the teaching of science. National
Science Council of Canada, Ottawa.
Rourke, L.,
Anderson, T., Garrison, D. R., & Archer, W. (2001) Methodological Issues in
the Content Analysis of Computer Conference Transcripts. Journal of Artificial
Intelligence in Education 12, 8-22.
Sutton, C.
(1992) Words, Science and Learning. Open University Press, Buckingham.
Wenger, E.
(1998) Communities of Practice: Learning, Meaning and Identity. Cambridge
University Press, Cambridge.