AI ‘not yet ready’ for testing without humans
Published 26/02/2024
English language testing providers are using Artificial Intelligence to enhance the experience of students, but maintaining a human touch within exams is still important as technology develops.
Speaking with The PIE News, VP of Product Innovation & Development at ETS, Kara McWilliams, said that the big challenge for test takers has been how to prepare for tests.
The personalised solution allows them to target spots that most need improving, she said.
“TOEFL is the gold standard in testing and we want to elevate test takers to achieve the best they can,” McWilliams told The PIE.
It comes after the company recently launched a personalised, AI-powered test prep experience, TOEFL TestReady.
“We have a real customer obsession at ETS and it is about investing in customer [success].”
ETS is also looking at solutions to solve challenges around the corner that test beyond cognitive skills of learners and test takers.
Senior research manager at Cambridge University Press & Assessment, Carla Pastorino-Campos, recently said that while the sector shouldn’t be afraid of AI, it is “essential to understand more about the risks associated with ethics, integrity and quality”.
It is a point also picked up by McWilliams, who said that the team at ETS is also constantly working on how AI fits into its products ethically, and that it does not limit success of certain groups of students over others.
“We’re looking at how we can use AI and other groundbreaking technologies to enhance tests whilst preserving ethics, fairness and quality,” Pastorino recently said.
Additionally, a strong framework for AI-integrated tests in the future will “allow universities to continue making informed admissions decisions, ultimately shaping the future of higher education”.
Two ETS products, via the company’s ETS® Innovations™ arm, are already utilising AI in the interest of students.
The CONVERSE: Workplace app is designed to place users in an authentic immersive workplace, featuring ‘water cooler’ and work scenarios and virtual mentors.
Authentic Interview Prep, like the new test prep solution, utilises AI algorithms and gives personalised feedback on things like body language and confidence.
ETS is using filters on top of prompts from Chat GPT to create meaningful and appropriate context, McWilliams detailed.
Currently, ETS is the only English test with the unique AI-powered solution offering personalised test taker preparation, she said.
“The unveiling of TOEFL TestReady is important not only because it addresses a need we’ve heard directly from our test takers, but because we designed this with them as well,” said Rohit Sharma, SVP of Global Higher Education and Workskills, ETS.
But other providers also maintain a reliance on humans within testing.
In a statement to The PIE, founder and CEO of Password English Language Testing Ltd, Caroline Browne, noted that human involvement is “one of the factors that make our test results so reliable”.
“Whilst AI is a very important part of online testing in as far as online invigilation/ proctoring is concerned, we continue to rely heavily on humans to mark and review tests and also to analyse test data in order to ensure reliable test results,” she said.
Last year, director of Research – English at Cambridge University Press & Assessment, Evelina Galaczi, noted that technology is “shaking up the sector” and providing some “fantastic enhancements”.
Referencing a Mckinsey report, she said AI tools and Virtual Reality are offering innovations, Cambridge is not “rushing in and cut corners to suit the technology”.
The Linguaskill online English test uses hi-tech features but “keeps a human in the loop”, she said.
Like ETS, she said the actual use of AI within tests needs to be correct for English testing for university entry.
She suggested that some tests “prioritise technology and ‘on demand’ convenience over quality and relevance”, which can result in “poor performance, high levels of students dropping out of their courses, not to mention the anxiety and wellbeing issues associated with not having the English language skills [students] need”.
“We’ll still need to hold the ‘hand’ of technology for a smooth journey, as we evolve into the digital age,” she said.
The post AI ‘not yet ready’ for testing without humans appeared first on The PIE News.