Student Reaction to Video-streamed Content: Does it Enhance Knowledge and Understanding?

 

John Erskine and Marc Jones

Staffordshire University,

j.w.erskine@staffs.ac.uk

 

ABSTRACT

The present study examined student responses to the use of a video-streamed expert interview to improve knowledge and understanding in two key areas related to stress and sport performance. Following a series of learning activities that accompanied streamed clips, students were asked to identify what they liked best and least about the approach adopted, and state areas for improvement. A series of questions gauging reaction to both technical and pedagogical aspects of the video-streamed content was also undertaken. Overall, student responses were favourable to the use of streamed video in this context, with video streaming seen as being a more positive than negative tool in support of learning, especially in the areas of aiding clarity and understanding and presenting material in a non written format. Concerns highlighted included the non availability of text versions of the video clips and narrow access to the streamed material. The study identified areas for improvement which will allow for further development of this resource

 

Keywords

video streaming, learning, evaluation

 

INTRODUCTION and background

A key focus of many final year modules on undergraduate degrees is to get students to a ‘current understanding’ of the concepts and ideas being covered on the module by reading and understanding primary research materials. We wish students to have an appreciation of contemporary ideas, theories and research findings as opposed to a reliance on secondary source materials from textbooks that may be somewhat dated. The downside of this from a delivery perspective is that students can become increasingly frustrated at reading journal articles, given that this source of published material is often written in a very formulaic and stylistic manner. Certainly one might argue that the written format employed by many peer refereed scientific journals and the nature of language often used does not represent a user-friendly format from the student perspective.

As part of a Teaching Quality Enhancement Funded (TQEF) Project at Staffordshire University, SHE-STREAM (Sport, Health & Exercise – Streaming Teaching Resources for Education And Motivation) was developed partly in response to expressed concerns on behalf of students about the formulaic nature of material found in both primary and secondary reading materials. The aim of the SHE-STREAM project was to produce a stand-alone portal where examples of captured video content could be quickly encoded and stored on a server to be streamed to students on demand. In addition, the project also produced a first set of video-streamed resources to support a final year module entitled ‘Stress and Sport Performance’.

In terms of developing ideas for SHE-STREAM, and being a complete novice in the area of video streaming technology and pedagogy) a number of U.K. based video-streaming projects were examined for ideas and support. By far and away the most useful resource was the Click and Go Video project (http://www.clickandgovideo.ac.uk). Whilst the full ‘Video Streaming: a guide for educational development’ was not initially available when the SHE-STREAM project commenced, there were a series of resources that proved of immense value in setting up the SHE-STREAM portal and in developing ideas of how this new technology might be used in appropriate ways to derive learning outcomes. The SHE-STREAM project was in particular influenced by the Three ‘I’s framework adopted and outlined by Click and Go video project (see Young & Asensio, 2002). This framework involving 3 categories of image, interactivity and integration allowed decisions to be made regarding the approach to be adopted for the initial set of resources. Thus the focus of the SHE-STREAM portal was to ensure interactivity in terms of independent access, choice of clip and control over the material. Secondly, it was deemed necessary to develop a degree of integration by linking viewed clips with a series of Independent Learning Activites (ILA’s) which required students to extract information from the viewed clip, with all of this occurring within a Virtual Learning Environment (VLE). Therefore, at appropriate times, students were directed within LLS (Lotus Learning Space) by an active link to a series of clips that were streamed to them. As each clip was being viewed, students were required to answer a series of questions in an accompanying word document on the material viewed and then save this resource (and submit the completed ILA’s for assessment purposes).

The initial resource and the focus of this paper consisted of a 2-hour interview with Professor Lew Hardy of Bangor University, covering a wide range of research issues related to a number of the learning outcomes of the module. With respect to the Stress and Sport Performance module a series of questions examined two areas of importance – firstly issues surrounding the measurement of anxiety in sport and secondly a discussion of both catastrophe theory and research as applied to sports performance. Following the capture of this digital content, questions and answers from this interview were chunked into learning units, encoded and stored on a stand alone server to allow on demand access by students through the Lotus Learning Space VLE that supported the module delivery. For a more detailed account of the processes involved see Erskine & Jones (2003).

 

EVALUATION OF VIDEO-STREAMED MATERIAL

Formal evaluation of video streaming as a learning tool is in its infancy. The Click and Go video project used 3 case studies as part of their JISC/DNER funded project to assess the use of different approaches to video streaming within the fields of catering, surgery and fashion marketing using qualitative interviews to evaluate the value of video streamed content (see for example Zenios ).

Given the paucity of evaluation on video streaming from the users (students) perspective, a questionnaire was constructed which attempted to elicit (at both quantitative and qualitative levels) students reaction to the video streamed component of the Stress & Sport Performance module.

With reference to Section 14 – Evaluating the Educational Benefit of the Click and Go Video guide for educational development (Thornhill, Asensio & Young 2002), a series of questions were constructed to examine student’s reaction to central features and components of SHE-STREAM. Section A of the questionnaire adopted a quantitative approach (likert scale format) to examine areas such as ease of access, pedagogical rationale, video and sound quality, clip length, interactivity and perceived learning benefit. A copy of the question stems can be found in Appendix A. Section B of the questionnaire used a qualitative approach by asking students to comment in a written format on (a) what they liked ‘best’ about the use of video streaming on the module, (b) what they ‘least’ liked about the streamed content and finally (c) what ‘improvements’ they would recommend for subsequent delivery.

The questionnaire was administered to 70 students (male = 50, female = 20) in the final teaching session of the module in December 2003.  It was made clear to students completing the questionnaire that all answers would remain confidential and that their responses to the questionnaire would not be used in any way to influence assessment or grading on the module.

In order to analyse the data, responses to Section A were entered into SPSS (Statistical Package for Social Scientists, Ver. 11.0) to obtain descriptive statistical data.  In terms of Section B, to undertake an initial analysis of the data, written student comments were transcribed verbatim. Following this, the two lecturing staff on the module independently examined the comments made within each of the three areas and provided a list of categories that they considered represented emerging themes for the ‘best’, ‘least’ and ‘improvement’ comments. The independent emerging themes were then compared and an agreed list of themes selected for each area. Subsequent to this, each member of staff independently classified each written comment as belonging a given theme. Finally both lecturers came together to examine the consistency of their classifications of student comments across the three areas. Where comments had been differentially classified, discussion took place and agreement reached. In reality there were very few disagreements across the 3 question areas, between the two teaching staff.

With respect to what students liked best the following six themes were agreed by the teaching staff from written responses obtained and were labeled as: Clarity & Understanding, Differing Learning Format, Ease of Access, Additional Viewpoint, Interest & Motivation, and Assessment Role (see Table 3a).

Ten themes emerged from the student responses to what they liked least about the SHE-STREAM experience: As outlined in Table 3b these were: Access Issues, Question/Answer Relevance, Text Availability, Clip Length, Answer Comprehension, Excessive Time, Assessment Weighting, Controllability, Sound Quality and Nothing,

In terms of areas for improvement, again 10 themes emerged: Increase Access Areas, Text for ALL Clips, SHE-STREAM Design, None, Decrease Clip Length, Assessment Weighting, Additional Viewpoint, Improved Sound, Question/Answer Clarity, Headphone Provision (see Table 3c). It should be noted that there was a degree of correspondence between themes outlined in Table 3b and those in Table 3c.

 

INITIAL DATA ANALYSIS AND INTERPRETATION

A central aspect of the study was to investigate whether the SHE-STREAM clips and accompanying ILA’s resulted in an improved understanding of key learning outcomes of the module. Two questions in Section A were directly related to this area.  Over 70 % of students reported that the video streamed content had either ‘somewhat’ or ‘very much’ improved their understanding of concepts related to ‘Anxiety Measurement in Sport’. With respect to a second area concerning ‘Catastrophe Theory and Research’, this figure rose to 84.3%. This finding is supported by the qualitative data findings in Table 3a, with 46% of students commenting that the clips aided clarity and understanding.

For example one student responded:

“it helped me understand some concepts which were perhaps were unclear from reading the literature”

Students were asked if viewing of SHE-STREAM clips was required or essential to successfully complete their understanding of topic areas. Over half the students (53%) responded ‘frequently’ or ‘yes’, with only 10% stating that the viewing of clips was not essential for understanding.

In terms of a first use of SHE-STREAM within this module, these results are encouraging. Finally, it is pleasing to note nearly one in two of the students (44%) commented positively on the different nature that the streamed content represented as an alternative to text based material. One respondent commented

“it was very different from reading the work and made it a lot more clearer and enjoyable”

 

Table 3a   Positive Emerging Themes and Illustrative Quotes

 

As highlighted by the Click and Go video project, streamed video offers the opportunity of interaction with the learner in terms of access (asynchronous), choice (selection) and control (start, stop, pause and review). From student responses it appears that a significant number of students felt that interaction was an important and helpful aspect of the SHE-STREAM approach. For example, far more students (> 80%) felt either ‘somewhat’ or ‘in total control’ of the technology, as opposed to 17% feeling ’somewhat’ or totally’ controlled by the technology. One in five students (21%) highlighted ease of access as a key positive aspect of SHE-STREAM (see Table 3a).  As one student stated,

“we were able to take the information in when we wanted and we could take it in at a pace that suited our needs”

With respect to control of the streamed clips, 73% of students reported using the pause button either ‘frequently’ or ‘all of the time’; 75% used the cursor to drag a clip backwards to repeat a phrase or sentence, but perhaps somewhat disappointingly only one in five students (26%) used the cursor ‘frequently’ or ‘all the time’ to repeat a clip in its entirety.

None of the students reported the Question and Answer clips as being too short, one fifth of the students thought the clips too long, whilst over three quarters (78.6%) thought the length of clips about right. Clip length emerged as a negative theme (see Fig 3b), with 9 comments (13%) offered supporting the view of clips that were too lengthy. However given the advanced nature of many of the issues addressed, it would have been difficult to break down often complex answers into shorter units.

A number of students commented that being able to hear an explanation of issues from a source other than the lecturer on the module was of benefit with one student suggesting that,

“it enabled us to gather another lecturers viewpoint which enhanced the chances of learning”

 

Table 3b   Negative Emerging Themes and Illustrative Quotes

 

Turning to emerging negative issues, by far the biggest issue that students commented on (46% - see Table 3b above) was the fact that the streamed clips were only available (initially) to a single room (a departmental I.T. room). This was a decision undertaken due to technical reasons early on in the project. Not surprisingly this issue was identified as a priority for improvement (see Figure 3c) with 23% of student comments identifying this as an issue that require addressing for the next delivery of the module

For example, one student stated “allow clips to be accessed from all rooms/facilities in the university”

A major concern identified by students was the fact that,

“text was not available in most of the topic areas”

The Special Education Needs and Disability Act (SENDA) highlights the importance of providing alternative ways of obtaining video and audio materials. Unfortunately the time demands present within the project did not allow for all clips to have accompanying text – again this issue was high on the agenda for improvement (Table 3c) with 14% of students identifying this as an area to work on.

Nine of the 70 students commented negatively on clip length (see Table 3b), and this again emerged as an area for improvement with a similar number of students (n=8) suggesting that for example we,

“possibly break down the longer answers into more manageable clips, to further aid understanding”

Although there is some suggestion that video segments in multimedia should be limited to twenty to thirty seconds (Alessi & Trollip, 2001), the length of clips really depends on the content and how it is being used. With respect to the content area and the ILA that was required to be undertaken, it is felt that clip lengths of much longer duration are justified in this instance.

Not surprisingly a few students felt the work required to complete the ILA’s was not sufficiently rewarded in assessment terms (full completion of 5 ILA’s was worth 5 % of the module grade). However this was very much a minority with only 7% of students suggesting increased weighting of the ILA’s as an area for improvement. Just over half the students (56%) reported that ‘most’ or ‘all’ of the ILA’s were undertaken at the time requested, though it was disappointing to find that when requested to view a set of web resources accompanying a series of video clips explaining elements of catastrophe theory, 83 % of students reported not undertaking this activity. It is felt that this is primarily a design issue that needs addressing to allow for a more overt and seamless structure to link video to web resources within the VLE materials produced.

Finally, a number of students suggested improvements to the interface between ILA’s and streamed clips within LLS, with a request to,

“label questions with week numbers by them so students are aware what question is required for each ILA”.

 

Table 3c   Improvement Emerging Themes and Illustrative Quotes

 

CONCLUSION

While very much aware of the non-validated nature of the questionnaire used in this study, initial analysis of the student evaluation of SHE-STREAM is more than encouraging. Given that this was the first delivery of an untested system, it was pleasing to note that the overall volume of positive comments exceeded those offered identifying negative elements of the SHE-STREAM approach. The majority of students reported that the video streamed material was beneficial to their learning and understanding of key concepts and that the streaming format offered a welcome alternative to text based materials. The goals of achieving a resource emphasizing interactivity and integration appear to have been partially met. On the negative side, students wished that the streamed material was more widely accessible and had accompanying text for all the streamed clips. A minority of students had some concerns about the design of SHE-STREAM but these were not fundamentally problematic. The questionnaire has highlighted those areas to address to further improve the pedagogic value of video streamed content in support of the learning outcomes of the module.

 

ACKNOWLEDGMENTS

Thanks to Sharon Gwinneth and Sue Howlett for their technical expertise during the development of the SHE-STREAM Project. Thanks are also extended to Marc Jones in initial data analysis and comments on an earlier draft of this paper.

 

REFERENCES

Alessi, S. M., & Trollip, S. R. (2001). Multimedia for Learning: Methods and Developments (3rd Ed.) Needham Heights, Massachusetts: Allyn and Bacon.

Erskine, J.W. and Jones, M. (2003). Video-streaming in sport and exercise science: Is an image worth a thousand words?  LINK, Health, Leisure, Sport & Tourism LTSN, 8, 23-25.

Thornhill, S., Asensio, M. and Young, C. (2002) Video streaming: a guide for educational development. Retrieved January 23rd from http://www.clickandgovideo.ac.uk/

Young, C. and Asensio, M. (2002). Looking through the three ‘I’s: The pedagogic use of  streaming video. In S. Banks, P. Goodyear, V. Hodgson and D. McConnell (Eds.) Networked Learning 2002, Sheffield, March. Conference Proceedings, pp628-635.

Zenios, M.K. (n.d). The use of video streaming in higher education: A report on the evaluation of the click and go video case studies and the educational benefit for learners. Retrieved 26th April, 2003, from the Click and Go Video Web site: http://www.clickandgovideo.ac.uk/