NLC2004 /Proceedings / Symposia / Symposium2
Analysing the pedagogical value of video treatment and text in a digital media application
Sheffield Hallam University
Abstract
An Evaluative Case Study of a postgraduate induction programme on CD-Rom containing video examines the pedagogical potential of video and the synergy between video treatment and text. Although the work is still in progress, some useful patterns have emerged. This paper looks at three types of evaluation approaches to collect user feedback on the value of video in order to show that the CD-Rom has made a difference to learning experiences. It also examines the possible relationship between video treatment, text and student motivation and explores the notion that despite the use of a degraded ‘digital’ image, video can be used to generate moods and create a journey of learning for a learning experience and that the experience can influence learning as much as visual richness in educational multimedia.
Keywords
Video treatment, educational multimedia, pedagogic screenwriting, independent learning, teaching and learning, postgraduate learning.
INTRODUCTION
The evaluation work underpinning this paper is partly
developmental and partly summative and the use of video, integrated with text
is partly based on the resources developed within the JISC/DNER Click and Go Video Decision Tool, (Thornhill, Asensio and
Young 2002) and The Three 'I’ framework of Image, Interactivity and
Integration. This practitioner based framework strikes a chord with the
educational design of a postgraduate induction programme on CD-Rom which
contains video and text. Although the Click and Go project team refer to the
educational benefit of making a subject more appealing, more fun and engaging through video, the potential to
experiment with the visual literacy is boundless and the students' emotional
experiences captured in this inquiry may have potential benefits to
practitioners.
The design and contents of the postgraduate CD-Rom 'Mastering University' draws upon the theoretical framework provided by the Click and go Video project and the pedagogical potential of screenwriting (Koumis, 1991) and the perception amongst learners that video is "easy" for cognitive processing (Salomon, 1984). It also attempts to draw on the rules relating the use of narration, animation and text on screen and the retention of information in multimedia learning (Mayer, 2001). Beard and Wilson (2002) use the power of experience and emotion to engage the learner whereas Merrienboer et al (2003) make more use of instructional learning theories such as problem solving and scaffolding to control cognitive loading. The presenter/tutor featured in many of the video chapters in the CD-Rom, has himself written on the value on experiential learning and developed ways to engage learners more deeply using a variety of techniques that does not solely rely on learning content. His work describes how accessing deeper emotions can be difficult and an unknown territory, but the belief is that the stimulation of senses creates an affective response which in itself intensifies learning. The concept of creating a journey of learning within the CD-Rom has without doubt been a factor in the design of the multimedia application.
The aim in the case study was to use the CD-Rom, Mastering University to evaluate the way that video treatment has been applied and to analyse its influence as sets of learning experiences which have been transferred from a live face to face event to the CD-Rom. The results of the case study research and the way video was transferred to multimedia attempts to assess whether the synergy between text and video has worked in a simple interactive form and how it has affected learners' emotions. The results also showed how this pedagogical approach served the learner and how it provided sufficient conditions to affect deeper processing of content. The purpose of the evaluative case study and its use of video treatment might support a number of ideas about the future design of educational multimedia.
The CD-Rom Mastering University uses Macromedia authoring tools and QuickTime playback software to combine video and text as a way of introducing new learners to postgraduate study in a 'journey of learning'. The use of video to instruct learners in the skill of writing and thinking critically might not be a new concept, but the use of video and treatment to 'hook' and 'prime' learners in an attempt draw the learner into less stimulating material is used. The results of formative evaluation of the CD-Rom have helped to guage the potential of video and video treatment and the educational impact of using video in a digital application. However, the value of using video as; an 'image', as a way of 'integrating' media and as a way of making the learning 'interactive' is relatively new and the tentative conclusion from the study so far is that the combined effects of all three maybe far wider than we think and this research is intends to show how video treatment as opposed to lengthier pieces of video may have a bigger impact on learning than we had imagined.
The case study: 'mastering University'
The pedagogical design of the CD-Rom draws upon the experiences of both the multimedia development team and the academic team. The instructional design pays special attention to the way the learning environment is able to support students from different educational backgrounds and distance learners who were unable to attend the two day induction programme. This is done in a variety of ways; using video to capture the atmosphere and content of the real event, using video and text to complement a range of different learning styles, using video to introduce learning tasks in order to create a form of learning experience and using text and video to create a journey of learning. The content and structure of video work, sometimes referred to as video treatment, helped to create the idea of a journey of learning within the CD-Rom and video format or style of video was used to convey information and learning content. These are categorised as: observation of classroom activity, priming for a task, one-to-one tutorial, hooking the learner and viewing for pleasure. The material can be viewed in linear or non-linear mode, with no fixed entry points. The CD-Rom provides approximately one to two hours worth of learning. It offers the learner an opportunity to interact with the material with the intention of providing an engaging learning environment and a way of accessing video and text files. The CD-Rom contains 15 separate video segments across eight chapters. The chapters are; Introduction to CD, Coffee and Papers, What is Mastery? Writing Introductions, Literature Review, The Learning Buzz, Writing Conclusions and Reflecting on Your Experiences. The video element is a strong feature of the CD-Rom and shows a range of recorded classroom activity of students from a prior cohort attending the postgraduate induction programme in September 2001.
The formative evaluation of the CD-Rom in 2002 used a small number of postgraduate students and academic staff to assess the value of the CD-Rom for distance learning and e-learning. The respondents applauded the use of video and responded very positively to video as a strength of the CD-Rom. The nature of the evaluation was formative and a superficial level of feedback on the use of the resource and on its impact on learning was collected. The evaluation did not supply any more detailed pedagogical reasons for its success. The existence, of what the design team refers to as the synergy between the video and text material was raised in the evaluation questionnaire. The results provided very little data for analysis and the findings could only conclude on the positive effect of video and on the right amount of video used. A new opportunity presented itself in 2003 for a thorough evaluation of the CD-Rom resource. This evaluation is still being done. Its purpose is to determine whether video treatment and the complimentary use of video and text had had a vital role in the CD-Rom and how video and text could be assimilated into learning for future applications.
methodology
There were a number of approaches in which the investigation could be tackled, but for ethical and practical reasons, an evaluative case study approach was chosen (Bassey, 1999). Evaluation methods differ and the ideas provided by Baker (2001) reflect the point about the purpose of evaluation and that evaluation should strengthen the confidence in what teachers do and secondly that first trials often show the failures of a programme or product. Mixed methods of gaining data were used because they work to provide the best understanding of the research issues (Creswell, 2003) and that they can be adapted (Binh, 1998). The research methods were designed around four themes. Each theme defines the context with which the students’ experienced or encountered the learning and the ways in which the topic could be investigated; a) Interactivity and user control b) Media Richness and complexity of Learning task c) Learner perceptions about the synergy between text and video d) Learner experiences and perceptions about the video treatment used to stimulate learning. The evaluation case study uses three types of evaluation method to explore the synergy between video and text. This paper concentrates on the themes covered by a) c) and d).
Objective-Led Type
The objective-led type used a questionnaire to evaluate the extent to which the CD-Rom had achieved its objectives. These objectives are summarised in the following ways and the questionnaire design reflects these four objectives; a) the use of video to engage the learner and provide information in a journey of learning, b) the integration of video and text to draw the learner towards the text materials, c) the interactive use of video to give the learner control over the resource and d) to produce an electronic version of the live event in order to give an opportunity to encourage not attending students to try out the same tasks.
Summative Type
The summative type was used as a way of gathering information and user feedback on how well the journey of learning idea was captured through the use of video and text. The questionnaire tool was designed in order to allow users to make a personal assessment of the media in terms of quantifying their perceptions about the amount of processed learning and to what extent they attributed this learning to a particular medium or the multimedia application itself.
Illuminative Type
The illuminative type was used to capture the participants' experiences and perceptions in a way that quantitative methods were unable to detect. This was achieved by obtaining qualitative data from students’ written coursework after attending the postgraduate induction programme. The learners conducted a written critical assessment of their own understanding of their learning styles, learning strengths and weaknesses. The results were also collected through Part D of the questionnaire.
Evaluation Strategy
The participants in the evaluative case study; a cohort of 63 postgraduates attending a series of lectures on critical reflection and research methods and a cohort 24 postgraduates attending a Postgraduate Certificate in Learning and Teaching in HE, were given a copy of the CD-Rom and a questionnaire. The participants were asked to work through the CD-Rom and then complete the questionnaire within a period of several weeks. The intention behind the use of two sets of students is to be able to compare and contrast the respondents' feedback for validity and reliability purposes. The students attending the Postgraduate Certificate in Learning and Teaching are experienced learners, and have a good academic background, whilst those attending the induction programme, tended to have mixed academic backgrounds. Both groups represented real and authentic contexts. This relates to the evaluation theory that proposes using real situations for issues of validity (Evaluation Cookbook, accessed 2003).
The questionnaires for both cohorts were designed to obtain learner feedback. A Likert scale was used to gain data on the way learners' rated their perception of the influence of media on their learning style with particular reference to the way video had supported their learning. The scale was also used to rate the impact of video and text at various points in the CD-Rom. The questionnaire was designed in 4 parts. This was done to reflect the different perspectives and learning themes that the inquiry needed to cover and to capture the users' perceptions. The third instrument used was a reflective critique. The reflective critique was an opportunity for students to appraise the role of the CD-Rom within their learning through an assessed piece of coursework and provide the inquiry with qualitative data. It was discovered midway through the data collecting process that due to the lack of reliable data obtained in the questionnaires, a 4th method was needed. The decision to include a focus group in the evaluation has been taken recently. The sample will be taken from those taking part in questionnaire and from those who wrote a reflective critique. Participants required for the focus group (not yet implemented) will represent a cross section of both cohorts. In order to take account of the range of educational experiences and cultural backgrounds that existed in the larger cohort, it was felt that all postgraduates attending the live lectures should participate in the evaluative case study.
questionnaires: Results
The results collected from the questionnaires failed to produce a large amount of reliable data but they did contain some useful evidence on the relative merits of the CD-Rom and on how well the resource had achieved its objectives. The four responses collected from those attending the postgraduate certificate in HE represented a much smaller return rate than expected (17%). The response rate from those enrolled on the Postgraduate Induction Programme was higher (50%), but any useful comparisons to make between the two cohorts need careful analysis.
First Cohort: Postgraduate Certificate in Learning and Teaching in HE
Despite the low return rate, the responses in the smaller group reflect a much greater level of accuracy in the way the questionnaires were completed, compared to the larger cohort. This may possibly due to the amount of time that the respondents devoted to the completing the questionnaire and their own personal interest on the value of multimedia as a teaching resource. In Parts A and C, all respondents paid tribute to the way video was used in terms of their own perception of learning and also indicated that video and not text had had the biggest influence. There were three questions relating to the respondents' perceptions of which medium had the most effect on their learning. In the first question, the respondents were asked whether video or text helped them learn to write at masters level; in the next question respondents were asked which medium helped to think at masters level and in the third question, which medium would help them most if they returned to the CD-Rom in the future. Overall, the respondents attributed 70% of their learning to video and 30% to text. (See Table 1).
Table 1. Students were asked to attribute in percentages, their perception about the amount learnt, to video and text. The figures below represent the mean for all four respondents.
|
Text |
10 |
20 |
30 |
40 |
50 |
|
70 |
80 |
90 |
100% |
||
|
Video |
10 |
20 |
30 |
40 |
50 |
60 |
70 |
80 |
90 |
100% |
Each of the eight chapters used video in different ways and in Part D, respondents were asked to mark or sketch on a graph, how they rated video as a learning aid for each chapter. The y axis represented a rating of video. The x axis represented video in each chapter and the different ways it was used. The four respondents indicated (in freehand) where they gave video a high or low rating. Three of the four graphs were very similar (see table 2). The Introduction chapter contains video to describe mainly the contents of the video and the purpose of the CD-Rom. The video element in Coffee and Papers is a classroom discussion and contains a low amount of learning content. Three of the four students gave the video a rating of 25%. In the next chapter, What is Mastery? the video contains instruction/teaching and a one-to-one tutorial on how to write creatively, critically and conceptually. Here, the rating for video was given 50% and in the other graph, it receives a rating of 75%. The next chapter, Writing Introductions, the tutor facilitates pair work and encourages students to construct an introduction to their written assignments. The video rating rises to 75%, but in the fourth graph it was given a lower rating. The Literature Review shows students working in groups in order to undertake a literature review under the tutor's guidance. Video is given a high rating by all respondents. The rating falls at the Studying Buzz and then rises to 50% at the chapters called Writing Conclusions and Reflection. Writing Conclusions contains some video of students undertaking a speed reading exercise and features the tutor giving advice on useful tips to summarise information. Reflection shows the tutor offering more guidance with a note on the power of reflection and a personal farewell.
Table 2. Students are asked to rate the video against chapters. The left graph represents the marks of three very similar graphs. The right graph represents the marks made on the fourth.
video rating
50% 50%
In Part B, the respondents were asked to identify some, if not all the different video treatments used in the CD-Rom. It is useful to note that classroom discussion and instruction/teaching by the tutor were considered to have engaged and motivated the learner most.
Second Cohort: Postgraduate Induction Programme
The responses collected from the second
cohort provided a much larger amount of data. Students participating in the
study were given the CD-Rom within the first few weeks of their course. In the
latter part of the semester 1, they were issued with the questionnaire at the
end of a lecture session. Participants were given 15 minutes in which to
complete it. Although there was a 50% return rate and Parts A and B were
completed reasonably well, there are concerns over the validity and reliability
of the data in Parts C and D. The completed parts here failed to match the
accuracy of the completed questionnaires from the cohort attending the
postgraduate certificate.
reflective CRITIQUES: results
The written critiques provided a substantial amount of qualitative data. Direct quotes, terms and words that refer to the value of video in the CD-Rom and used in the critique were used in the analysis. These references to the CD-Rom were seen as evidence of their learning.
It is interesting to note that some respondents expressed a cautionary note about entering into the inquiry and what amount of support the CD-Rom would provide for personal study. The following extracts typify the qualitative data collected and the statements contain evidence about the emotional engagement that the CD-Rom creates.
“The CD-Rom Mastering
University has had a tremendous impact in the learning process. First of all,
this is the first time in my education that along with the course material a
CD-Rom was given as a learning tool. The concept of learning to learn was
foreign to me. When the CD was handed over to me I was not convinced that it
would assist me in changing my learning style. But on going through it, it has
changed my entire outlook towards it.”
Respondents identified with the purpose of the resource and the value of interactive features. This gave control of learning to the user and an opportunity to experience the learning in different ways.
“One of the strengths
of the CD-Rom is that it allows you to see and hear the tasks in action and
then encourages you to do the task yourself. It is much easier to remember
something that you have seen live than just recalling words on a page”
“The CD-ROM Mastering
University has been essential. It is a simple to use tool, communicating
theories and practice in an interesting manner. The format that incorporates
text with video clips is effective as not only did I learn from the initial
reading but all of the information was mirrored through video.”
Some students were able to identify with the concept of integrating text and video.
“Hence, the video
clips gain more attention and leave a long lasting effect on the mind. The
clips demonstrate the text given, and hence it is easier to understand the text
by linking it to video clips. Thus the CD-Rom is very beneficial for distance
learning students as well. Since all this information is organised in a compact
disc, it is easy to carry and view anywhere and anytime.”
“I thought the CD-Rom
was a great idea and it also made me go through the different tasks on the CD
when I arrived home. Apart from the positive aspect of being something
different and challenging to normal books, the great thing was your one to one
seminars with your own tutor, my tutor. If the CD had just been packed with
only text files and no videos, would I have used it? I very much doubt so.”
There were some very strong indications that Coffee and Papers and What is Mastery? were memorable and supported their learning. Although there were references to all chapters, the references to Coffee and Papers and What is Mastery? were overwhelming. Students were also able to express an emotional engagement with the CD-Rom without feeling the need to learn.
"It was an added
bonus to be able to reprise the lectures in a one on one situation as well as
being there with the teaching group. The text and video complemented each other
well, whilst the graphics are effective without being too fussy and detracting
from the main function of the CD-Rom. Concepts are explained clearly and there
is an informal yet business like feel, due in part, to the National Park
setting which induces a sense of well being."
"In the Coffee
and Papers session in the CD-Rom, the author also mentions......' The idea is
to understand and experience how our own relaxed alertness helps us to create a
positive mental state. The mind and the surrounding ambience are
important'...Therefore finding a comfortable and quiet space is what I
need"
CONCLUSION
The success of the CD-Rom might be attributed to the use of video treatment and the way text and video had been integrated and also to the way the presenter/tutor had development team had designed and adapted the materials to produce an electronic version of the live event. There was an issue over the reliability and validity of the data in the questionnaires taken from the larger cohort and the way in which respondents had failed to complete the questionnaires. However, the results provided an insight into way learners saw the role of video and this gave a rough indication as to where the medium has had its greatest influence. The summative approach concentrated on the role of video and text to provide a number of learning experiences that might include elements of interactivity. The illuminative approach provided possible links between the results from the four returned questionnaires from the first cohort and the evidence supplied in the critiques. The qualitative results suggest that students received a set of learning experiences. These experiences range from motivation to learn, to the control the CD-Rom offers to be able to replay the video more than once or to the consolidation of information. The instructional design of the CD-Rom in terms of its contents, use of video and interactivity supported a number of learning styles and this offers an insight into the engagement value of video and whether the learning styles suited one form of video format or treatment over another. There is a strong indication that video helped students to assimilate learning and that video was used for surface learning and the text was used to consolidate information. The large amount of positive descriptive words and terms used by respondents to describe the overall impact of the resource was very encouraging, but only a small proportion specifically mentioned the learning value of integrating video with text. In the reflective critiques, the evaluation contained a high number of references to Coffee and Papers and What is Mastery? against the relatively low numbers referring to Writing Introductions, Literature Review, Writing Conclusions and Reflecting on your Experiences. Coffee and Papers contains a light amount of learning content, whereas What is Mastery? is heavily steered towards content and the teaching of a number of concepts. A tentative conclusion might suggest that learners need to have lighter material to absorb first as a way of a learning 'hook' before tackling harder material. In the questionnaire, the video element of Coffee and Papers (classroom discussion) and What is Mastery? (teaching/instruction) both contributed to motivation and engagement of learning. The relative importance of the responses given in Part B and the references made in the reflective critique show how beneficial to learners it was to see a classroom discussion style as well as the instruction/teaching style of video. It seems that both were effective at engaging and motivating learners. It now raises questions about the relationship between a) the engagement and emotional cognitive factor of video, b) its content and c) student learning styles. This is something that the instruments were unable to detect, neither is it clear how much the tutor's teaching style and video content influenced results. However, this can provide a useful line of inquiry for the ensuing focus group.
SUMMARY
The methods used in Evaluative Case Study and the guidance given in three ‘I’s framework, attempted to link a number of theories on learning and media to attempt to conclude on the usefulness of the ideas expressed, but also to further the development of practitioner based knowledge. The Evaluative Case Study investigated the pedagogical value of compressed video, video treatment and the integration or synergy between video and text. The evaluation tended to concentrate on learners' perceptions about the use of video and text. The study of learning experiences under the sub headings of interactivity, engagement, stimuli and learning from instruction in an evaluation based inquiry revealed how learners felt about the role of video and text through a number of evaluation approaches. The objective led approach yielded a small amount of data on which it seems that a limited amount of interactivity did have a positive influence. The influence of video treatment to create a sense of a journey of learning and to help learners engage with text based resources may help practitioners using video in multimedia to either apply these ideas or to develop a practitioner based understanding of the theory behind the practise.
ACKNOWLEDGMENTS
I wish to thank Colin Beard, Senior Lecturer in the Faculty of Organisation and Management at Sheffield Hallam University for his continuing support and advice for this research and the collection of data for this document.
REFERENCES
Bassey, M (1999) Case Study Research in Educational Settings. Open University Press, Buckingham.
Baker, E (2001) Technology: How do we know if it works? In Heineke, W, F and Blasi, L. (Eds) Methods of Evaluating Educational Technology, IAP, 5, 73-83
Beard, C, and Wilson, J, P (2002) The Power of Experiential Learning: a handbook for trainers and educators. Kogan Page, London.
Binh, P (1998) Quality Evaluation of Educational Multimedia Systems. Australian Journal of Educational Technology, 14, 2, 107-121
Clark, E (1994) Media Will Never Influence Learning. Educational Technology Research & Development, 42, 2, 21-29
Creswell, J. W (2003) Research Design: Qualitative, Quantitative, and mixed methods approaches. Thousand Oakes, London, Sage.
Kuomis, J (1991) Narrative Screenwriting for Educational TV. Journal of Educational Television, 17, 3, 1991
Mayer, R (2001) Multimedia Learning. Cambridge University Press.
van Merrienboer, J, J, G., Kirschner, P, A., Kester, L (2003) Taking the Load Off a Learner's Mind: Instructional Design for Complex Learning. Educational Psychologist, 38, 1, 5-13.
Morrison, K (1993) Planning and Accomplishing School Centered Evaluation. Peter Francis Publications.
Salomon, G (1984) Television is "Easy" and Print is "Tough": the Differential Investment of Mental Effort in Learning as a Function of Perceptions and Attributions. Journal of Educational Psychology, 76, 4, 647-658
Evaluation Cookbook, 2002: Visited: Nov 2003. URL; http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html