Not many people will have missed the recent hype regarding the impact of ChatGPT on the education sector since its launch in November. It is a debate which the Password team and our colleagues at Examity, our online proctoring partner, have been following with keen interest. For us the key question is whether online exams are more vulnerable than traditional paper-based versions to this kind of development.
Whilst the pandemic accelerated the adoption of online assessment, paper-based exams, such as the UK’s GCSEs and A levels sat under rigorously enforced conditions at spaced desks in large halls patrolled by trained invigilators, have not disappeared. Nor are these types of assessments invulnerable to technology-based attempts to cheat, for example sneaking a smartwatch or mobile phone into the exam hall.
How does this compare to high-stakes assessments in a securely proctored, online environment? Responsible online test providers will in fact operate in a very similar fashion to GCSE exam halls For example, for the Password Skills Plus test which is used to assess the English language proficiency of university applicants, the same set of rigorous procedures in place. For example, during the exam, candidates are required to meet exacting standards relating to authentication (ID), test administration (attire, equipment, environment) and exam conduct.
Together with invigilation rules, these procedures are used to maintain exam security and detect deception and any improper practice. In the case of Password Skills Plus, the ratio of online invigilators to students (one to four) is far higher than the one in thirty typical for paper-based exams conducted at official exam centres, making it easier to catch any attempts to cheat.
Exams run in a securely proctored online environment also have other security features.
Again, with Password Skills Plus, AI monitoring is also employed, with exam sessions recorded and data stored. This makes spotting potential infractions much easier. Examity’s invigilators can respond to incidents during the examination session but in addition, events flagged on the time-stamped tracking video can be fully investigated by multiple reviewers retrospectively, before an exam result is awarded.
In fact where language proficiency assessment is concerned, ChatGPT has some very particular limitations. It is not able to imitate second language learners by taking into account mother-tongue interference. In other words, it has not (yet) learned how to make deliberate mistakes which would lend authenticity to its answers.
Ironically, it is its facility for writing flawless text, which is its downfall.
Dr Helen Wood is head of school partnerships at English Language Testing Ltd.
