The arrival of the latest English Proficiency Index (EPI) from Education First (EF) has led to the normal splash headlines in the English language press around the world.
Some of them are completely predictable:
“Germans score very high” (The Local, Germany)
“Singapore most proficient in English among Asian countries” (Mothership SG)
Others are a shock:
“Lebanon dropped 20 places in English Proficiency in just one year” (Lebanese media outlet The 961)
Can a whole country really forget its English in just 12 months?
It seems unlikely. More probably there was a glitch in the sample.
EF is based on a large global sample of test results, but how big are the sample of individual countries? Small samples can throw up unexpected biases, so IELTS does not release test results on small countries like Lebanon because of small sample size.
TOEFL does include small countries with small samples and, like EF, is a highly reliable test: so, if an unlikely EF finding is replicated in TOEFL there is more evidence to support the EF finding.
Take Lebanon. In the EF test, it ranked 21 places below Spain; TOEFL has it just one point behind. Yet both tests have Singapore at number one in Asia and Germans have very high scores in both.
Although EF publishes its rankings by country, its analysis concentrates on regions and demographic groups. Its finding that Latin America’s English is still rising fast among adults does not seem strange, nor that their children’s score has dropped since the pandemic.
Indeed, EF’s findings that across the world school-age children’s scores have dropped 50 points on average due to Covid replicates similar findings for English language learning in US schools, as does its hypothesis that remote learning and wearing masks in class may be contributor factors.
However, for the second year running EF results showing that male test-takers are outscoring females stands in contradiction to the results of every other language test. According to one professor, females score higher in all of them.
Which doesn’t mean EF is wrong. It may simply be reaching a different female demographic. In order to ensure its findings are reliable, we need to see these results replicated.
So, are men pulling ahead of women on language learning? Is Lebanese English collapsing?
To know for sure, we need a global test based on a large representative sample of each country’s population. For that we will have to wait till 2025, when OECD launches its first Foreign Language Assessment as part of the PISA suite of tests. It will only tell us about school children, but it will tell us about all the school children in that country, not just those who choose to go online and take the EF test.
