AI, invigilation and (online) assessment
Published 27/01/2023
It has been impossible to avoid the flurry of articles regarding the impact of ChatGPT on the education sector since its launch in November.
Only last week, the PIE’s Nicholas Cuthbert made the argument that ChatGPT had “sobering implications for traditional models of teaching and assessment”, the key concern being whether such tools compromise the academic integrity of all assessment.
It is a debate which the Password team and colleagues at Examity, our online proctoring partner, have been following with keen interest. So, how is cheating on assessments changing, and to what extent are online exams more vulnerable than traditional paper-based versions, particularly to innovations like ChatGPT?
Whilst the pandemic accelerated the adoption of online assessment, paper-based exams, such as GCSEs and A levels sat under rigorously enforced conditions at spaced desks in large halls patrolled by trained invigilators, have of course not disappeared.
Nor are these types of assessment invulnerable to technology-based (as well as other) attempts to cheat, for example sneaking a smart watch, mobile phone or other devise into the exam hall.
However, the ultimate threat of having all exam marks voided if a miscreant is caught, as well as the presence of invigilators at a ratio of one to every 30 students, helps to deter and detect cheating.
How does this compare to conducting high-stakes assessments in a proctored, online environment? Responsible online test providers engaged in this domain will in fact operate in a very similar fashion to GCSEs and A levels exams.
Using the example of the Password Skills Plus test which is used to demonstrate that international university applicants have the level of English language proficiency required to fulfil their conditional offer, we can see the same set of rigorous procedures in place.
For example, during the exam, candidates are required to meet exacting standards relating to authentication (ID), test administration (attire, equipment, environment) and exam conduct. Together with invigilation rules, these procedures are used to maintain exam security and detect deception and any improper practice.
In the case of Password Skills Plus, the ratio of online invigilators to students (one to four) is far higher than is typical for paper-based exams conducted at official exam centres, making it in fact easier to catch any attempts to cheat.
“Not only can cheating be caught during the examination session, it can also be caught afterwards when exams are reviewed”
In addition, exams run in a proctored online environment have certain other security features.
Again, in the example of Password Skills Plus, both human and AI monitoring are employed, with exam sessions recorded and data stored. This makes spotting potential infractions much easier. Just like in-person exams, Examity’s invigilators can respond to incidents during the examination session but in addition, events flagged on the time-stamped tracking video can be fully investigated by multiple reviewers retrospectively, before an exam result is validated or voided.
Caroline Browne, Password’s CEO, concludes, “In the case of securely proctored online exams, not only can cheating be caught during the examination session, it can also be caught afterwards when exams are reviewed.
“For Password exams, the security of human invigilators plus AI tracking, followed by Examity’s auditor review, is combined with checks by Password’s markers who investigate flagged incidents, and cross reference a candidate’s written and spoken performances with their results for other skills.
“These elements are all overseen by our Academic Management team, making it incredibly hard to cheat and get away with it.”
Of course, as stakeholders, it could be argued that we (Password and Examity) have an interest in playing down the extent to which ChatGPT could revolutionise cheating. But, where language proficiency assessment is concerned, it does have very particular limitations.
It is not able to imitate second language learners taking into account mother-tongue interference. In other words, it has not (yet) learned how to make deliberate mistakes which would lend authenticity to its answers. Ironically, where languages are concerned, it is its facility for writing flawless text, which is its downfall.
About the author: This is a sponsored post by Helen Wood, Head of School Partnerships at Password, an online testing company specialising in assessments of academic linguistic proficiency for international users of English in school, college and university settings. She is a former Head of International School and Head of EAL and a graduate of the bilingual international relations faculty (IHEID), Geneva University.
The post AI, invigilation and (online) assessment appeared first on The PIE News.