Transforming English Language Assessment Using Technology, with telc: Language Tests

Note: Surpass Assessment is now the company name for BTL and BTL Surpass.

In this case study, discover how telc Language Tests have utilised functionality in Surpass to:

• Transform and enhance a traditionally paper assessment for on-screen delivery.
• Create real-world authentic tests appropriate to the learners of today.
• Use multiple item types to test different skills.
• Engage learners through use of rich-media.
• Innovate in language testing design and delivery.

What has been achieved

The telc English A2-B1 School to Job examination aims to provide school leavers with an additional language qualification set in a workplace context, thereby giving them a clear advantage and increasing their chances of employment.

Rather than simply moving paper test content to on-screen, telc saw the opportunity to make improvements to the candidate experience and the assessment development process.

An innovative digital development team was created within the business, specifically to drive the project forward.

As well as assessing typical language skills such as speaking, reading, writing and listening, the new digital assessment allows for a new category of items specifically designed to assess mediation skills – highly important in the workplace. Recognising that young learners are not always being prepared for life in a digitised world, telc’s goal with this assessment was to produce an examination more closely related to how students live and learn. The assessment is also fully integrated with the new CEFR descriptors, the international standards for describing language ability.

Due to the functionality available in Surpass, this test is unique in its ability to test multiple skills within one item, transforming the English Language testing market.

The content of the examination, mapped to international standards for describing language ability is a valid and reliable measurement of skill. telc’s thorough quality assurance and standard setting processes ensured that the digital test content was equivalent to that of the paper test.

How it was achieved

By using intuitive item types available in Surpass, telc have created engaging media-enhanced items to replicate real-world applications of language for the 21st century workplace.

Examples include responding to a text messaging chain, and listening to and interpreting directions to find a location on a map. In recognition of the flipped learning approach being increasingly adopted by teachers, an emphasis was also put on the inclusion of video, a format pertinent to the examination’s target group, and a better reflection of how they live and learn. Just some examples of the use of video include watching a tutorial before answering a series of questions, and in a new approach to creative writing tasks, candidates watch a video with no sound and use their mediation skills to write a narrative.

A series of design thinking workshops and engagement with experts and key stakeholders was an essential part of the thorough product development process. Representatives from BTL were also invited to attend some of these sessions, which gave a great insight into their processes and allowed BTL to provide the best support to telc during this project and beyond. telc demonstrated great understanding of the importance of having the right people on board, and interviews, workshops and surveys with students, teachers, examiners and test experts as well as internal stakeholders formed an intrinsic part of the process. This also served to turnaround the opinion of those doubtful of this new way of testing.

As part of customer-oriented product development and telc’s quality assurance measures, there was an intense piloting phase, with the usability, delivery, and newly developed content tested with a total of 700 middle school students in Germany. Data was collected, which compared some of the existing content that had been transferred directly from paper to on-screen with the newly developed items, in order to check that items were performing as expected.

This data also helped to prove that language skills can be effectively tested on-screen, to those wary of a new approach.

With all items evaluated against international standards, telc could have confidence that the new digital test was comparable to its paper equivalent.

How it was received

The assessment has been met with positive feedback from test takers. For them, it has proven to be an enjoyable, modern, and speedy format.

For prospective employers, it is an accurate and realistic test of the skills required in the workplace, giving the school-leaver a desired advantage and the employer the necessary assurance.

Not only does the enhanced content give candidates a better testing experience, but moving to digital testing also brings with it logistical advantages, such as removing the need to securely transport large quantities of paper to and from test centers, and quicker evaluation turnaround times, saving money and resource.

telc’s willingness to share their knowledge and experiences with digital testing with the Surpass Community and the wider language testing community has gone some way towards encouraging others to take their first steps on this journey.

In summary

telc Language Tests effectively demonstrated how by utilising functionality in Surpass, assessments can be transformed, for the better. They have brought language testing into the 21st century by testing in a way which matches how students learn, rather than goes against it.

telc English A2-B1 School to Job puts English Language testing into a new, digital context and makes the connection between education and the workplace.

The resulting certification demonstrates language skills, workplace skills and computer skills, giving students the best possible start to their future careers.

Watch more

In this presentation from the 2020 Surpass Conference, Sean offered the Surpass Community an insight into the developments made with telc’s popular language testing programme over the past year.