top of page

Video items in e-Assessment – what happened?

Video is a now a key factor in personalized learning, affords learners access to educational sources how and when they like it. The e-Assessment world tends to shrug its shoulders when asked why video hasn’t really take off for high-stakes e-testing and why e-Portfolio (using video as evidence for qualifications) is not quite into the early/late majority adoption lifecycle in many areas.


Almost every buzz-term in the education technology field (flipped classroom, MOOC, BYOD, personalized or social learning) has video as a key component. Proponents such as Michal Tsur say that video is “the glue that ties our new learning methods together and facilitates them in the first place”.


Research from the Teachers College at Columbia University over 20 years ago claimed that students who were exposed to the integration of video content in their learning, outperformed non-exposed peers on exams. A 2012 report by Cisco claims that video accounts for more than 50% of all mobile traffic, and will reach at least 66% by 2017. So why is video the ‘shy kid in the corner’ for e-Assessment?


If we consider that for much of e-Assessment (and especially e-testing), there is a cost shift from delivery (essentially shifting paper) to the creation/ production of items and tests or retooling of processes for greater efficiency. For video items, this means that item writers and qualification development professionals need to consider different skills, which are not traditionally in their skill set, such as storyboarding, lighting, scripting, location finding etc.


While digital cameras and phones have transformed our ability to capture memories, special occasions, or other events that were not possible even 10 years ago, video fit for high-stakes (or ‘meaningful’) exams still has a number of barriers to overcome. Here are five to consider:


1. Face validity does the test look like a real test and for the intended purpose? Many e-Assessments still rely on paper conversions (text and image), fewer take a ground zero approach and consider the real potential of media for testing that are just not possible via simple text and picture e-Assessment items.


2. Scalability How easy is it to build a sustainable item bank that can withstand exposure and maintain relevancy to the curriculum? Some commentators in the e-learning world lazily call e-Assessment ‘cheap’. Consider the use of video in the UK Driving Theory Test: 130 1 minute film clips to sustain a test volume of 1.5 million exams per year puts the cost of bloated e-learning systems to shame.


3. Use of Non-Standardised Hardware A fundamental principle of e-Assessment is to produce a test that is presented in a standardised way for all candidates, so that no candidate is unfairly disadvantaged. For example, video reproduced using different frame rates by non-standardised hardware in a test centre, can create significant differences for candidates sat in the same room (e.g. http://boallen.com/fps-compare.html).


4. Security Holding items on dispersed, local servers can create security issues, especially with video files being many times the size of a text-based multiple choice question. CAPS compliant laptops, as users with enterprise-wide encryption and anti-virus software will testify, can severely hamper performance.


5. Access to Broadband Test preparation for video items may seem easy to set up from an exam creator’s point of view. ‘Flat’ video files served up and embedded on a website, or linked to You Tube can help show a candidate what to expect. However, broadband ‘not-spots’ mean that any practice test using video becomes a tiresome waste of time for prospective candidates, affecting their ability to perform to the best of their abilities.


Equally, the upload/ download of evidence for e-Portfolio away from a suitable (and free for

the student) corporate or educational internet connection can hamper a student in their pursuit of a qualification.


In July 2013, Jon Brodkin wrote a fascinating article for Arstechnica that tells us of how the never-ending argument between Internet Service Providers and video services (over how much one network should pay to connect to another) that directly impacts video streaming quality.


Needless to say, this turf-war compounds the increasing demand for better video quality, which exam creators appear to be reticent to engage with. Learner expectations of video quality have increased since the smudgy days of VHS (or 405 lines for our older readers), with Blu-ray, HD now prevalent in many Western households and 4k consumer hardware embedding itself. The growth of gaming over the internet has also raised expectations in terms of the user experience and the ability to interact (read: shoot-‘em-up) over broadband.


Needless to say, an exam board that has little experience of using video to test candidate abilities will inevitably steer clear of a radical change in curriculum design, pedagogy, skillsets and untried methodologies.


While organisations such as the Serious Games Institute at Coventry University have made

considerable strides in the use of gaming and education within computer generated environments (e.g. soft skills, career guidance), the high-stakes e-Assessment e-testing and portfolio worlds are conspicuous by their absence.


Outliers such as the UK Driving Test still continue to be beacons for using video. A study on Hazard Perception Testing (HPT) was performed by TRL in 2002, which indicated that while there a number of research procedures for measuring hazard perception of candidates, none was deemed suitable for licensure.


The HPT test was deemed (at that time) to be statistically reliable, and that adequate training of learner drivers could help them achieve hazard awareness closer to that of experienced drivers. The test was introduced in November 2002. The creation of film clips that offer face validity is costly, and fraught with operational difficulty.


Interestingly, the recent move to convert 130 of the HPT clips to CGI/ animation not only removes the use of film, but offers more control over environments, vehicles and characteristics of movement within each clip. While the test hazards remain the same, and are still relevant, contemporary hazards can be introduced at reduced cost.


The tipping point for the use of video maybe reached with Bring Your Own Device (BYOD). Candidates and learners are invited to use their own hardware to take their test, which is subject to meeting minimum technical criteria, and perhaps coupled with rendering and/or security software.


As issues with proprietary formats being used over the internet are slowly being overcome with the use of tools such as HTML5, developers and exam creators will no longer be able to ignore the bow wave of video in everyday learning – the exam must reflect and align with how the training/ learning was performed.

Comments


Commenting has been turned off.
bottom of page