Pearson’s Freya Thomas Monk, SVP English Assessment, tells Ron Ragsdale about the changes afoot in EFL testing
Can you tell me a bit about your role and your previous experience?
Freya: This new role was created to recognise the growth and investment in the area of English Assessment. I’ve been with Pearson for 14 years. The first job I had was as a strategy adviser, and I worked on the business case for PTE Academic. I then moved around a bit, in schools, strategy and operations, and then back into English language assessment two years ago. Although I now lead a distinct Assessment part of the business, I don’t see it as being a standalone product. It links seamlessly with learning. The Pearson objective is to help students learn English, and learning and assessment form a holistic approach. Our teams are based around the world, which gives us more empathy for our diverse community of learners. We’re very conscious of the problems that beset testing in different contexts.
PTE Academic is a single product. How does it work for such a diversity of learners and contexts?
Freya: PTE Academic is one product, but it is tailored to different levels. It produces a score on the GSE (Global Scale of English) between 10-90, which is aligned to the CEFR from A1 to C2. Although it is not adaptive, the items are served up randomly from a very large item bank, so each test will be unique. There are twenty different item types, which are designed to be flexible, to match the proficiency level. We also use real video and audio, mimicking what people will experience in real life. The test is digital, but it is administered in secure test centres. You have a headset to interact with the test. Each individual is tested individually. There are no cohorts, so you can book it whenever you want to take it – up to 24 hours in advance.
How does the test get marked, particularly for productive speaking and writing?
Freya: The automated system uses artificial intelligence to assess writing and speaking. This system has been designed and validated against hundreds of thousands of human responses. We get scores back very fast, 87 per cent within 2 days, but lots on the same day. The student gets a full-page report, with the overall score and a breakdown by skill, and then highlights for each area, such as spelling, grammar, fluency, etc. So, they know what areas they need to work on.
Where does Pearson feel they are at with regard to use of AI in this area?
Freya: I feel we are ahead of the game. We have been doing this for nearly 10 years now and have been improving it continuously. We’re pretty sophisticated in our understanding of what it can do. For example, we are often praised for a lack of bias. Our machines are familiar with all accents around the world, whereas a particular human examiner might not be able to understand some accents. We are removing the bias, so that you can get at the underlying proficiency. Also, human examiners get tired, distracted or hungry. This can be a real problem that automated marking avoids.
“You can book it whenever you want to take it - up to 24 hours in advance.”
How about using AI for formative assessment?
Freya: We already do! We understand how we can use this in the new products being developed by Pearson English. The latest is English Benchmark Young Learners – a tablet-based product. It’s a different type of test for younger learners, but under the bonnet, it’s some of the same technology. We are also looking at developing more products that use automated marking, such as the Versant English Test for the professional training sector
And for PTE are you seeing growth?
Freya: Yes – in 2017 we grew 67 per cent in actual test takers and this level of growth is continuing. For competitive reasons, we can’t share the actual numbers of tests administered. A key factor, however, is the increasing recognition of PTE Academic in various countries. We were recently renewed with the Australian government for their visa programme, for example. We have 96 per cent acceptance in the University sector in the UK and Canada, 100 per cent in Ireland, New Zealand and Australia. In the US there are so many institutions, and each one makes its own decisions, but we’re making good progress. We are always looking at new opportunities.
Do you conduct studies exploring the effectiveness of assessment using automated marking?
Freya: Yes, we have an annual call for research – in fact we have just opened our 2019 call. We do this with external research groups and we also run correlation studies ourselves internally, and we publish research on our website.
