Multimedia tools and their accessibility
Case study prepared by Dr Jo Badge, Web Resources Development Officer, School of Biological Sciences, University of Leicester
Based on a pilot study conducted by Jo Badge. Alan Cann, Jon Scott and Emma Dawson. Funded by the Fund for New Teaching Initiatives, University of Leicester.
Introduction
With the increasing use of online learning resources in Higher Education comes a need to evaluate these resources in terms not just of accessibility but also usability. Many resources may include accessibility features but it is how the students employ these and if they are useful that will be essential to its successful use to increase student understanding.
At the University of Leicester we have three tools for producing multimedia online resources: Macromedia Breeze, Impatica and Macromedia Flash. Macromedia Breeze provides a set of tools for online communication and is a networked solution which can produce stand alone or networked resources. Impatica is a stand alone product. Both tools are primarily used to enhance PowerPoint slide presentations by adding notes, visual navigation controls and audio. A third tool, widely used for presentation of rich media online, is Macromedia Flash.
Our internally funded project aimed to evaluate and compare these three products in terms of their SENDA compliance (HMSO, 2001) for student users who may be required to view the resulting files as part of their undergraduate courses. A pilot study was set up to create teaching materials using each of these tools. Two groups of undergraduate students tested these materials to compare the three products. All from the Faculty of Medicine and Biological Sciences, one group of students all had registered accessibility issues, such as dyslexia and/or learning difficulties and the second group provided a matched control with no registered accessibility issues. Our aim was to concentrate on the usability of these products from the student perspective.
Methodology
The project required the production of learning materials using three different formats – Flash, Breeze and Impatica. In order that fair testing could be carried out the same content was used for all three styles of output. Several PowerPoint presentations containing animations were found within the teaching materials presented in the School of Biological Sciences and Medical School and were made available by the kind permission of their authors. One presentation used was from a lecturer at Trinity College, Dublin (permission granted by author). After some initial testing, two of these presentations were selected for use in the trial and each was converted to Breeze, Flash and Impatica with a narrative sound track.
ii) Recruitment of users for testing
Current undergraduate students were recruited for user testing. The University’s AccessAbility centre provided information about the body of students with registered accessibility issues in the Faculty of Medicine and Biological Sciences. They provided assistance in writing directly to those students registered with them to solicit volunteers for the project. Email was used to notify the whole undergraduate population within the School of the project and to seek volunteers who may have had accessibility issues that they had not declared or who had chosen not to register witht the AccessAbility centre.
Students were offered a small cash fee for completing the task. Ten students were recruited and came individually to user testing sessions. Each subject was asked to complete a form notifying us of the nature of their registered accessibility issues. Seven of the subjects were registered with dyslexia, one with dyspraxia, one with a visual impairment and one with a hearing impairment. There were two males and eight females from a range of years of study and courses within the Faculty. A control group was recruited once the subject testing was completed. Recruitment was via email and we received a huge response. This enabled the selection of a matched group of students with the same gender, course of study and year of study as the test subjects.
iii) User testing
Students attended the testing sessions individually. They were asked to view each of the presentations and complete a series of questions about their content. They were video taped and observed by a member of staff while carrying out the tasks. The questions were content driven, and therefore gave the appearance of testing the user’s knowledge of the subject, while in fact they were designed to stimulate the use of the presentation software.
User testing sessions were carried out in a private office. The sessions were run by one member of staff and who also took notes and video taped the session. Students were asked to sign a consent form to give their permission to the video taping of the session. The video tapes were used solely for data collection so that timings, actions and questions could be reliably recorded and later re-analysed.
The PC used for testing was running Windows XP and was the machine used to create and test the materials, to ensure that all software, visual and sound problems had been identified. A simple html page was created to host six links to the test materials. There were six presentations to watch. Two presentations were chosen, one on Immunology and a second about DNA replication and these were each converted in the three formats: Flash, Impatica and Breeze. It was decided to present these in the following order to all test subjects:
1. Immunology – flash version
2. Immunology – Impatica version
3. Immunology – breeze version
4. DNA replication - flash version
5. DNA replication – Impatica version
6. DNA replication – breeze version
With only 10 test subjects and 10 control subjects it was felt that the sample size was too small to allow us to vary the order in which the presentations were viewed by each subject.
The test subjects were given minimal information about the project and the tasks. They were asked to answer a series of questions about each presentation which were designed to encourage them to make use of the features of the three different platforms. Questions were based on the subject of each presentation and therefore appeared to test the student’s knowledge of the subject (e.g. immunology) rather than their use of the platform. Questions were designed such that they addressed information contained in the text of the slides, audio track or some graphical or pictorial element of the presentation.
Test subjects were also asked to comment on the three different styles of presentation once the test was completed.
The same audio track was used for each type of presentation. The Flash versions of the materials provided a simple ‘next slide’ link to click forward through the presentation. There was no ‘skip forward’ or ‘rewind’ facility; it could only be viewed in a linear fashion. The Breeze and Impatica presentations both had ‘vcr’ type controls to allow the user to play/pause, skip forward/ back through the presentation and a list of slides so that the presentation could be played from a chosen starting point. They also contained a transcript of the audio track, displayed in time to the voiceover. All three styles of presentation contained the same level of animation within the content slides (see example figures)
Figure 1: screenshot from presentation on Immunology using Flash
Figure 2: Screenshot from presentation on Immunology using Impatica
Figure 3: Screenshot from presentation on Immunology using Breezeiv) Data
The video taped sessions were analysed in depth for various factors relating the test subject’s use of the three platforms.
Factors analysed:
The video taped sessions were analysed in depth for various factors relating the test subject’s use of the three platforms. This entailed watching each user testing session carefully and using the tape timer to measure the length of various activities, and counting the number of times certain actions were carried out by the user.
The answers given by the students from the task questionnaires were marked and their scores analysed.
Ten test subjects were recruited and tested individually. Seven of the subjects were registered with dyslexia, one with dyspraxia, one with a visual impairment and one with a hearing impairment. There were two males and eight females from a range of years of study and courses within the Faculty. The control group was matched for gender, year and course of study.
a) Features used
Records were made of which features of Breeze and Impatica were used by the subjects. These features related to any operation that could be performed such as using the play or pause button, clicking on the tabs in Breeze to view a script or different menu view or using the slide selector in Impatica. We found that the test group used significantly more features than the control group (Fischer's exact test, p < 0.001).
b) Scores
In order to assess learning, the question sheets completed by the subjects were marked and the number of correct answers totalled. However, we found no statistically significant difference between the scores obtained by the two groups (t = 0.379(18), p > 0.05).
c) Pointing behaviour
It was noticed that some subjects used a pen, finger or mouse arrow to point to the words they read information on screen or on paper. This is a strategy sometimes recommended to and used by dyslexic students to help their reading. In this study, there was no significant difference in pointing behaviour between the test and control groups (Fischer's exact test, p > 0.05).
d) Use of search facility
Breeze and Impatica both provide a search facility that can be used to search for keywords or phrases within the text of the slide or script. Some students made good use of this feature to find answers to the questions they were asked to complete. However, when the data were analyzed, there was no statistically significant difference in use of search facilities between the test and control groups (Fischer's exact test, p > 0.05).
e) Time taken to complete the task
Initially it was assumed that the test subjects might take longer to complete the tasks and view all the presentations that the control subjects. However, in this study we found that there was no significant difference between the overall time taken by the two groups to complete the exercises (t = 1.431(18), p > 0.05).
f) Use of navigation tools
Several navigation tools exist in both Impatica and Breeze similar to VCR controls and those common to internet media players (play, stop, pause, fast forward, rewind). A note was made of how many slides played before the subject made use of any one of these controls. In our study, the test subjects located and used these controls more quickly than the control group and this difference was statistically significant chi-square = 0.000 (df=1, N=2), p < 0.05).
Discussion
This project aimed to inform the use of multimedia software for students with accessibility issues. Although all three products conform to a greater or less degree to web accessibility guidelines, a formal analysis of the way that they were used by students was just as informative as the formal guidelines.
We showed that students with accessibility issues use this type of multimedia software differently from the control group and this may impact on their level of learning. The test group made much more extensive use of the alternative content views and visual navigation schemes that allowed them to customise their viewing of the presentation. For example, they would pause the presentation so that they could make sure they had read all the material on each slide before continuing to the next. They turned the sound on or off. When answering questions about the content, they used the slide lists to scan through the slides. Several commented that they liked the visual of a thumbnail picture of each slide in the outline list of one package, as many had a visual memory of the way the slide looked rather than a recall of its title.
Overall, the test subjects reported that they preferred to customise their use of these materials and therefore preferred the presentations offered with the possibility for customisation for this reason. One test subject with visual impairment commented that the video controls and colouring used on the Impatica background were too small to see and it was not possible to alter the size or colour of the text and controls.
The preparation of materials for use in this project provided some unexpected outcomes. This was an area of work that took considerably more time than estimated and had several false starts as we were forced to change track when technological problems were not surmountable in the time available. Ultimately it was discovered that animation in PowerPoint did not transfer simply to Breeze or Impatica but required some careful and detailed alterations to ensure that there were no noticeable differences between the final versions.
To gain statistically significant results on such a small scale project was surprising but very encouraging. These data provide an insight into the uses that students make of multimedia software and indicate that the ability to take control over them was welcomed. Where significant differences between the test and control groups were recorded, there is no evidence that the test group was disadvantaged by the software used.
References
Jakob Nielsen's Alertbox December 5, 2005: Talking-Head Video Is Boring Online(accessed 10 July 2006)
HMSO (2001) Special Education Needs and Disability ActRetrieved 23 August 2006
