The truth about online testing

Published 12/05/2020

It seems that a new online English language test is launching almost every week at the moment, which is hardly surprising given the extremely challenging circumstances faced by “traditional” assessments due to the coronavirus/ Covid-19 pandemic.

This rapid introduction of so many new tests has led to some interesting discussions and critiques of online tests and testing, some aspects of which get confused and need unpicking.

“At Password we are under more scrutiny than ever before, which to be honest, is no bad thing”

At Password we are under more scrutiny than ever before, which to be honest, is no bad thing. Since we have been delivering online tests in partnership with hundreds of universities, colleges and schools since 2008, we are pretty confident about what we have to offer.

The questions that are most interesting and challenging for us and indeed for all test providers (both paper-based and online) concern invigilation practices (also known as proctoring) and of course good old-fashioned cheating.

Validation

Before we get to that though, any institution thinking about adopting or accepting any English language test needs to ensure the test design was informed by a sound usefulness framework. Bachman and Palmer’s framework is commonly used and means that the test needs to have reliability, construct validity, authenticity, interactiveness, impact and practicality.

For laymen such as myself (Password tests are academically managed by professor Anthony Green’s world-class experts in English language testing and assessment at CRELLA) this means that test results should be reliable for the purpose they are being used for and that test takers should be able to easily understand how to take the test.

Delivery Method

Now, we come to delivery method – paper-based or online. There are pros and cons for each of course but for me, the raison d’etre for the development of Password testing was the security, convenience and accessibility of online testing.

A test can be randomised with a large bank of questions and run 24/7/365 anywhere in the world that has a reasonable internet connection. It can then be marked instantaneously by computer or remotely by trained markers accessing test results and reports also from anywhere in the world.

This contrasts with the need to create new tests for each and every paper-based test session, securely distributing the mass of papers and then somehow returning completed scripts to markers – time-consuming, error-prone and risky, and costly.

“I think that often when people think about online tests, they confuse the method of delivery with invigilation”

I think that often when people think about online tests, they confuse the method of delivery (paper-based or online) with invigilation (test taker being observed or not).

So, for example, an online test can be delivered in a test centre and invigilated by people just as a paper-based test often is. Conversely, a paper-based test can be taken at home with no invigilation just as an online test can be. In the case of Password, our high stakes tests are always invigilated.

Invigilation

Next, we come to invigilation itself. Invigilation for online tests (and any other type) can be undertaken in person or remotely (online).

Again, using Password as an example, until very recently our high stakes tests have all been invigilated in person. As we work with universities, colleges and schools whose institutions are well used to running high stakes tests and exams, we allow them to invigilate tests where and when they require following our procedures.

The Password team also review each and every test taken back at our HQ in London – this is again an advantage of online testing as we can see results, reports and data within seconds and look for any anomalies. We don’t find many, but when we do the institution is swiftly informed and can take appropriate action.

Given the necessity of continuing to assess applicants for admission to courses starting this autumn, along with the difficult circumstances in which we find ourselves, at Password we have developed what we are calling “at home” testing to help our institutional partners. These tests are delivered and invigilated as usual by our partner institutions but this time the invigilation is undertaken remotely online.

Again, our partners follow our procedures to remotely access test takers’ PCs to invigilate their tests along with best practice guidance for delivery in challenging locations such as China where internet speed and bandwidth (not the “great firewall”, at least for Password) can cause problems.

After checking the test takers ID and asking questions pertinent to their application to ensure the identity of the candidate, the invigilator watches both the test taker and their screen as the test taker works their way through the test.

Another way of doing this for institutions without the capacity to invigilate remotely themselves is to use the services of our partner ProctorU to invigilate for them. In this case, ProctorU provides human invigilation backed up by artificial intelligence monitoring and of course we continue to review results as well.

Next, we come to online test invigilation without any human involvement using a locked-down browser and some form of artificial intelligence to deter test takers from cheating which also records the session either by video or thousands of photographs.

“Discussions about online testing almost always centre on cheating and catching cheating test takers”

We have looked into this as it would seem to be a very neat and cost-effective solution but to date have not found a reliable enough way to ensure security except by people reviewing hours of video after the testing session has finished which would seem to defeat the purpose of eliminating human invigilation. I hope this is one we will crack in due course but no provider has it right yet in my opinion.

Now, we come to what I think a lot of people think of as online testing. Tests being taken with no invigilation either in person, remotely, by lockdown browser or anything at all. This would never ever be recommended for high stakes testing be it paper-based or online, but somehow it always seems to confuse the discussion when it comes to online testing.

And finally spare a thought for the poor test takers. Discussions about online testing almost always centre on cheating and catching cheating test takers.

In reality, most test takers don’t try to cheat; trust me, I’ve personally reviewed enough tests! And can you imagine how stressful being watched taking a three-hour exam online is.

It’s bad enough taking a driving test which mostly has a low cost of failure, but the online test may be a student’s one chance to achieve their ambition to study their chosen course at their chosen institution overseas.

Remote invigilators can interrupt at any time, shut down the test at any time, and even after the test has finished they can invalidate the test results taking away the test taker’s dream. Of course, we must remain vigilant for the few and put robust barriers in their way, but let’s not this derail the need to help the many deserving students wanting to access our world-class institutions this autumn.

Caroline Browne is founder & CEO of Password English Language Testing. She began her working life as an English Language teacher in Singapore and then later Japan and went on to run a language school in London. Caroline joined the university sector in 1992 and has had senior management roles in both the public and private sectors. In 2008 she founded Password, which provides secure online English language testing in 120 countries.

The post The truth about online testing appeared first on The PIE News.