It’s EF EPI season and newspaper headlines around the world are once again either celebrating their country’s success or bemoaning its failure in the ranking based on the results of EF’s free online language test, EF SET.
“Worst in the EU?” asks The Local, France, bemoaning the country’s performance on the English Proficiency Index, to give EPI its full name.
“Slipping English Proficiency?” sighs the Philippine Star.
And from Lebanese media outlet The 961, “Lebanon dropped 20 places in English Proficiency in just one year.”
The EF Index is annual and its eye-catching outliers always grab the headlines. In reality, of course, the entire population of a country rarely improves its English in a single year, nor does its proficiency normally plummet. EF doesn’t suggest it does – its own analysis focuses on regional and demographic trends.
In terms of individual countries, what often changes is the number and nature of the individuals who choose to take the free online test known as EF SET on which the rankings is based. If half-a-dozen Austrians took the test, their results won’t tell us much, as a very small sample of a population can tell you very little about the country’s English proficiency.
And if 200,000 Dutch people took the test, but nearly all of them were university students, we still wouldn’t know much about the rest of the Netherlands. If a large sample is not representative, then it is still not a reliable indicator.
Because of the sample size problem, IELTS only reports the results from its 40-50 biggest markets. Small countries, like Austria, ranked number three by EF, or the Netherlands, at number one, do not figure at all in IELTS statistics by country.
EF boasts a very large sample size worldwide, but we have no data on the EF sample size of individual countries, nor do we know how representative of that country that sample is.
We do have information on gender. In 2022, around 22% more females took the EF SET test, which provides the Index data, than men. Astonishingly, for the second year in a row, the males out-scored them. Yet females do better than men in every single other international language test. Has EF discovered a major change?
Possibly. However, EF SET tests only listening and reading, and misses out the productive skills – speaking and writing – in which female candidates traditionally over-perform. Additionally, it uses only multiple choice questions (MCQs), a test format which is both objective and reliable, but which has been shown to significantly be an advantage to male test-takers.
The EF index also shows us the type of countries which do well. They are generally small, mostly European and Germanic language speakers, while countries which don’t dub films and television beat those with the same or similar languages who do.
If we compare the 2022 EF results with those from another international exam, TOEFL, taken in 2021, we can see that many of the same countries and types of country appear in the top 20, though sometimes in a different order.
TOEFL also doesn’t release national sample sizes, but since it is used by professionals for immigration purposes, it includes results for Anglophone countries as well.
TOEFL scores for different English-speaking countries can vary quite dramatically over time, especially those for small countries. The last time the Gazette analysed the scores in 2020 using, from memory, the 2019 results, Ireland came top with 100 points, just ahead of the Netherlands on 99.
In 2021, the average score for Irish candidates was 94. So far the Irish media is not running headlines screaming that the level of English in the country is in decline.
TOEFL top 20 countries by total score 2021
Overall scores in brackets.
1. Austria, Malta, Trinidad and Tobago (102)
2. Slovenia, Switzerland, Germany (100)
3. Netherlands (99)
4. South Africa, Singapore, Belgium, Denmark, Luxembourg, United Kingdom (98)
5. Canada, Iceland,Sweden (97)
6. India, Croatia, Estonia, Finland, Hungary, New Zealand (96)
