Can Multi-Modal assessment sweep away the AI froth?

If economists believe AI will mirror the dotcom boom-and-bust of the early 00s, how should the assessment sector ready itself? Environmental and corporate control concerns aside, how can its impact be simultaneously be leveraged and mitigated?

It’s tempting to quip that AI and its advocates are boorish, aggressive pub bores or bar-room lawyers (“like a not very good expert at more or less everything”). Real/ true AI assessment deployments at scale remain elusive, bar some early examples in test item creation, camera usage within remote proctoring, and edge cases for extended response marking. So is it just froth, or are there pragmatic decisions to make?

“Exam owners shouldn’t be discouraged by the boorish, non-expert AI advocates.”

Exam owners shouldn’t be discouraged. As AI tools wreak havoc on assessment instruments (such as take-home essays) and capabilities quickly evolve, pragmatic strategies are needed. So, how to manage these new (digital) risk vectors? Answers are hiding in plain sight.

Exam regulators already approve over a dozen assessment instruments, many already digitised. Using a basket of these ‘multi-modal’ assessment instruments, exam owners can pragmatically evolve their assessment palette.

What can comprise Multi-Modal Assessment? Many of the instruments in the table below can be judiciously AI-aided in creating, delivering, and authenticating assessment journeys. Dive into Ofqual reports or exam owner qualification specifications, and you’ll see how some qualifications already use up to ten instruments. Here are 18 that are already recognised.

Multi-Modal Assessment

Why go multi-modal? It mitigates the risks of one-shot high stakes exams, one-time malpractice, and the scourge of AI-powered written scripts destroying qualification currency. And multi-modal also offers an holistic view of the learner’s capabilities.

Why is multi-modal not being talked about? The assessment discourse is dominated by high-stakes school exams. The only assessment currency that fly-by-night policy folk, politicians, and pundits know – their own lived experience. This is despite many qualifications taken in and outside school with multi-modal. The notion of “(re)introducing coursework” usually leads to sneers and jeers, shutting down everyday experience from the assessment front-line.

“Multi-modal is hidden by people who can only relate their own lived experience: written exams in a hall.”

What does this mean for exam owners? A key strategic task is to develop a clear target operating model that is multi-modal and encompasses the best of digital with AI. That encompasses a variety of supervised assessment instruments to diffuse and dilute the risks. This doesn’t mean dissolving high-stakes exams. The ability to recall knowledge and understanding under time pressure at speed is rarely appreciated. But you can bet aircraft passengers, surgery patients, and construction workers do.

“Exam owners need a target operating model with a variety of supervised multi-modal assessment instruments – including the best of digital with AI.”

What should suppliers do? The days of singular software or services platforms are on borrowed time. Integrations are smoother than the days of daily CSV file transfers. Some believe software will be reduced to system prompts; the AI middleware decides whether to create a temporary User Interface, power-up a service, or gather data. Then retrieving from a basket of awarding Standard Operating Procedures for the user to choose from.

Instead of an exam admin conforming to what the software dev thinks you need, the software adapts to the user’s needs. Make your own workflow, not what the supplier thinks it is. True flexibility.

“Software and solution suppliers must serve exam owner SOPs. The days of ‘our way or the highway’ in software are on borrowed time.”

Investors? Assessment was a polycephaly beast before AI. And this will remain so. But investors who can corral the mission-critical elements that supervised assessment needs will prosper. And supervised is the key word. Assessment, just like investment, lives or dies by public confidence. Unimpeachable provenance, no more black boxes, and a catalyst for lightening-quick go-to market impact to maximise returns. True enterprise-level digital transformation that embraces multi-modal isn’t for everyone. Investors know that under-capitalised businesses can quickly fall behind, while exam owner customers need to compete for learners.

“Under-capitalised, mono-solution businesses surviving on organic growth will fall behind.”

Multi-modal assessment follows the same principles as developing diverse working groups: increasing objectivity, reducing risk, maximising outcomes. The best solution providers, exam owners, and investors will reap rewards by avoiding the froth, and leveraging the power of off-the-shelf multi-modal, aided by a sprinkling of AI.