After a packed couple of days of presentations, workshops, and a unique virtual networking experience, day 3 of the Surpass Conference didn’t disappoint.
The day began with a selection of special interest groups covering a range of topics including accreditation of programs using online proctoring, the impact of COVID-19 on formative and summative assessment in schools, and AI in marking. These sessions were a great way to get delegates talking about some of the key topics for the industry, and provided a good opportunity to connect virtually with like-minded colleagues – wherever they were in the world. They really helped to bring together the similarities that exist in assessment across our borders.
Delegates then rejoined the main ‘public’ conference stream for our last keynote of the event.
Liberty Munson, Senior Psychometrician at Microsoft Worldwide Learning spoke passionately about the ways in which technology can shape the future of assessment, and how technological advances are providing the opportunity to challenge the status quo.
‘The Cloud is fundamentally changing the way we interact with the world. Through the power of the Cloud, AI, machine learning, and associated technologies will fundamentally change how we think about skills validation and assessment design, development and delivery.’ – Liberty Munson
Liberty acknowledged how we’ve traditionally relied heavily of structured responses, but technologies such as machine learning, cognitive services and artificial intelligence are providing us with an opportunity to think differently about how we assess skills. She discussed how technology such as Automatic Item Generation allows us to rethink how we ask questions, changing item writing to focus on cognitive models and mapping.
During her presentation Liberty called for changes to the job task analysis to provide a data-driven understanding of the skills required in a job role and what we should be assessing, and a greater consideration of customer needs to create assessment experiences people want.
Liberty ended her presentation by asking us to imagine a world with no more tests, at least not in the traditional sense we’re used to. Whilst the objective evaluation of skills will always be necessary, Liberty suggests we need to be evaluating skills as someone is performing a job, or as close to that as we can get.
‘We need to expand our definition of assessment to one that is a more integrated learning and assessment experience that combines frequent planned low and high stakes assessments that not only evaluate competency but guide learning.’ – Liberty Munson
Our final community story was presented by Romana Moss from the Wine and Spirit Education Trust (WSET) who gave an insight into how the organization’s plans to roll out computer-based testing were disrupted by COVID-19, but led to a rapid implementation of remote invigilation.
Romana took the community through WSET’s process for paper exam delivery highlighting the need to move to CBT, before outlining the plan to pilot and go live with their first on-screen qualifications to establish a best practice.
With plans to continue pilots disrupted by lockdown, Romana highlighted how the organization went through a forced change of introducing remote invigilation in a short timeframe, before taking us through how that process works for WSET, and the variables that needed to be considered to give their APPs and candidates the correct support.
Romana finished by sharing WSET’s key learnings from the implementation of remote invigilation, including the importance of robust communications.
This was a fantastic insight into one organization’s experience of adaptation due to COVID-19 and the successful implementation of remote proctoring in difficult and unexpected times.
An insightful panel discussion closed the event, with some words of wisdom from guests Romana Moss, Liberty Munson and Amanda Dainis.
The panel considered whether the industry is likely to be more open to change as a result of the pandemic, before finishing with their advice to the industry going forward. Amanda Dainis summed up thoughts by commenting: “Try to look at the silver lining of the disruption that’s happening now, and allow it to be a foundation to push you to make changes later, even after this is ‘over’.”
Thank you to everyone who tuned in and took part over the three days for our first virtual conference, which despite being a little different to our usual event, was certainly a resounding success thanks to the fantastic contributions from the Surpass Community.
You can catch up on all of the presentations now over on the conference website.
