A Chinese Artificial intelligence (AI) platform has improved AI understanding of both Chinese and English, reports Yu Sun and colleagues from Baidu Inc., Beijing.
Up to now, the race to develop AI that can understand and use natural human language has been dominated by the US tech giants Google, Microsoft and Facebook – so the human language under focus has been English.
This international competition has a fast-changing leaderboard, reporting scores on GLUE (General Language Understanding), a set of tasks developed to test how well an AI really understands language. An average human scores 87/100 and Google’s BERT (Bidirectional Encoder Representation from Transformers) was the first AI to pass this milestone.
Then last December, China’s ERNIE (Enhanced Representation through kNowledge intEgration) became the first to score over 90, topping its US competitors. ERNIE, from Chinese Google-type giant, Baidu, built on BERT’s improvements – but with a twist.
As language teachers well know, understanding the meaning of language requires a lot more than understanding the meaning of individual words, and this has been a major challenge for AI systems.
One way that BERT trained to understand language was by hiding some words, then trying to predict what they would be by looking at the words before and after the missing word, a kind of word gap activity.
When ERNIE wanted to do the same it had to adapt the technique to Chinese. Individual Chinese characters don’t carry meaning in the way English words do. The meaning of individual Chinese characters depends on adjacent characters. So, ERNIE trained by hiding strings of characters so that it learned the meaning of pairs and groups of characters rather than individual ones.
This turned out to greatly improve ERNIE’s ability to understand English, too – because pairs and groups of English words also carry meaning that you cannot infer from the individual words alone, such as, ‘Harry Potter’ and common idioms such as ‘call it a day’. English language teachers may be surprised at the techies’ surprise: perhaps more of us should consult on these projects.
GLUE scores are now so high that a new, tougher test is being used. SuperGLUE includes complex open questions such as, ‘How do jellyfish function without a brain?’.
REFERENCE
- Sun, Y. et al. (2019) ‘ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding.’ Preprint for AAAI-20 (Association for the Advancement of Artificial Intelligence 2020 Conference, New York, USA). https://arxiv.org/pdf/1907. 12412.pdf
