As trade press gossip intensifies on Ofqual’s potential new interim leader, England’s exam
regulator has published a new report – looking at the barriers to using online and on-screen e-assessment for school exams in England.
In April 2021, it will be 17 years since Ken Boston, head of Ofqual’s predecessor, stood at London’s Royal Festival Hall and said “...we seek to prepare for the Secretary of State a comprehensive and deliverable proposal for implementation of e-assessment...and give a fair deal for learners." As 2021 emerges, are we no further forward? With hockey stick growth in case studies, successful deployments, and a vibrant vendor market, what’s the current situation?
Three from Twenty-Nine? The report mentions on-screen school exam deliver in just three countries: Finland, Israel and New Zealand. That’s a lot of absentees. According to the EU Eurydice unit, 29 European countries use e-assessment for at least one school level. Additionally, Australia’s NAPLAN, USA’s GMAT and SAT are missing in action.
16 Million E-Assessment Fans Can’t Be Wrong Equally, there’s no mention of non-school UK programmes. I estimate there are over 16M UK high-stakes exams delivered outside of schools per annum. Some UK programmes deliver up to 2.2M exams per annum. Last week, I tweeted about one programme in India that assesses 22M candidates via on-screen testing annually. Ofqual should engage and cite with these programmes – the body of evidence, already sizeable, is becoming overwhelming.
The report also omits the practical and pragmatic benefits of moving to e-assessment. This
includes moving exams out of crowded, unfit-for-purpose halls, and liberating school estate. The benefits of removing antiquated storage facilities – huge locked safes, barred windows – all just to keep paper secure. Never mind having email servers on site – how about big pallets of envelopes that schools struggle to keep secure?
Equally, there was a huge and timely missed opportunity on how e-assessment facilitates post-qualification admission (PQA) for university admission or further study – no more scanning and shipping, and critically, quicker results turnaround.
Retail and Office Space – There’s A Lot Of It About I was surprised at the report’s comment regarding test site availability and capacity. “There was not thought to be a supply of sufficient alternative site to be a practicable measure for England.” I believe this is untrue and unhelpful. It’s never been easier to locate venues for exams.
This isn’t just about London Excel sized venues that can seat 2,000 learners per sitting, used every year by financial education bodies such as ACCA and CIMA. For example, each English council borough has a public domain dataset of vacant commercial property. If Covid test centres can be deployed quickly with pop-up provision and modular build-outs, there’s no reason high-stakes e- assessment locations can’t be fitted out speedily and safely.
Covid and societal changes means that retail and office space availability is unprecedented. Ofqual should talk to logistics and property management companies, to develop scenarios and worked examples. Test centre fit-out is a lot cheaper than a department store.
We Want Our Halls Back The report makes much of the fact that school physical spaces are unsuitable for e-assessment. I’d go further. Halls should be used for teaching, learning, places of worship, and physical education.
Not exams. Exams of this nature need a secure, dedicated space that does not impinge on non-exam taking students. Removing this disruption and associated logistics/ storage from schools is entirely beneficial and with minimal downside – its omission from the report is odd. If we can take students to leisure centres, theatres, and other extra-curricula destinations, we can bus them to a professional, neutral test centre. Teachers would be delighted to ‘have their halls back’ during exam season.
Muck and Nettles On reflection after reading the report, it’s clear a ‘standardised’ approach and environment is non-negotiable. If you want a standardised experience, pay for a neutral exam venue. You wouldn’t expect a Big Mac to taste the same made outside of McDonalds restaurants – nor should we expect that for exam delivery.
Organisations source test centre sites every day, for exam owners around the world. No one likes to talk about the operational muck and nettles of delivery, but Ofqual should talk to those organisations. They are truly the unsung heroes of e-assessment delivery.
Never Enough A common stakeholder response on e-assessment is, ‘why can’t the government pay for it?’. About 20 years ago, I remember a government agency paying for just over 2,000 high-spec PCs to deliver a high-stakes e-assessment test.
The sector needs to learn this: IT provision for high-stakes exams in schools will never be good enough. BYOD isn’t an equitable solution for children. School budgets are always stretched. Business cases stand up if the tech is used on a regular basis through the year.
Removing Exam Risk From Schools The print, scan, on-screen marking and data analysis of paper exams are all out-sourced by the government, often via awarding bodies – why not the e-assessment tech and venues? Factor that into a procurement: let the vendor take the risk on using PCs outside of school exam season.
Companies offer leasing arrangements of computer kit or mass refurbishment of corporate kit. Or how about supporting UK plc? Commission Raspberry Pi 4 units, built in Pencoed, Wales? Ofqual should engage with these organisations and ask them: how many units, what support levels, what capex exposure is there?
Let those risks be transferred out of schools, so they can dedicate more resources and time to learning and teaching, underpinning the issue of public trust. It would also vastly reduce the burden upon educators to learn new assessment systems and regimes.
If, as the report intimates, that malpractice increases with e-assessment, then I fear that is a naive interpretation of on-the-ground practice. If a £10 note is stolen, the chances of recovery are next to zero. With a fraudulent credit card transaction, options are available. Predicting, detecting, and preventing malpractice with e-assessment gives us more opportunities. We will never eliminate risk from exam delivery. But e-assessment helps us to codify, quantify, and mitigate those delivery risks.
To initiate the move to e-assessment we need roadmaps on sun-setting paper delivery. Many exam owners have done this successfully at scale. Analogue to digital is not a dirty secret. Many IT companies do this as business-as-usual. Ofqual should talk to those organisations.
Headteachers – The Final Frontier? YouGov’s 2020 survey for Ofqual suggested it was headteachers who were the biggest opponents of change to e-assessment. They are accountable for exam results. If headmasters provide the exam venues at no financial cost, and sign the cheque for paper exams, exam boards will take the money. There’s no motivation to change. The power relationship remains.
The implications of moving power and control outside of the school gates, with neutral exam
venues and data insights, is never discussed in the public domain. If headteachers cede control of exam delivery, it also removes any potential opportunities for on-site malpractice.Neutral venues only want successful delivery – they have no skin in the game of the individual/ collective results. No admin staff or teachers need be implicated with access to exam papers. But that’s a difficult and potentially inflammatory discussion.
Stakeholders must engage deeper with school leaders on e-assessment; telling the stories on how e-assessment is prevalent once learners leave school. Learners are the biggest supporters of e-assessment for General Qualifications. Are these the same young people who take their driving theory test, citizenship test, key skills tests, and others, where they must pass an exam in order to join the workforce? All those tests, and many others, are delivered on-screen and have been for many years. Is it an existential threat to cherished school results, or is it the fear of losing control?
But how about a grander, strategic aim to road-test viability? Is e-assessment a vote winner? Could e-assessment help ‘level-up’ society? What benefits could reach the most number of voters? This isn’t about banning watches. It’s about providing true equity and fair access.
There’s a lot unsaid about using e-assessment in schools. I do understand why stakeholders seek the path of least resistance and disruption. Ken Boston’s vision, from the era of 3G mobile licenses and the third Harry Potter film, still stands up.
And, as Ofqual’s aspiring leader once said in an e-assessment presentation, "The interesting thing is that the technology exists now – to allow learning <and assessment> to be delivered on an ongoing basis in classrooms.”