dc.contributor.author | Ulčar, M | |
dc.contributor.author | Žagar, A | |
dc.contributor.author | Armendariz, CS | |
dc.contributor.author | Repar, A | |
dc.contributor.author | Pollak, S | |
dc.contributor.author | Purver, M | |
dc.contributor.author | Robnik-Šikonja, M | |
dc.date.accessioned | 2021-08-20T13:12:30Z | |
dc.date.available | 2021-08-20T13:12:30Z | |
dc.date.issued | 2021 | |
dc.identifier.uri | https://qmro.qmul.ac.uk/xmlui/handle/123456789/73681 | |
dc.description.abstract | The current dominance of deep neural networks in natural language processing is based on contextual embeddings such as ELMo, BERT, and BERT derivatives. Most existing work focuses on English; in contrast, we present here the first multilingual empirical comparison of two ELMo and several monolingual and multilingual BERT models using 14 tasks in nine languages. In monolingual settings, our analysis shows that monolingual BERT models generally dominate, with a few exceptions such as the dependency parsing task, where they are not competitive with ELMo models trained on large corpora. In cross-lingual settings, BERT models trained on only a few languages mostly do best, closely followed by massively multilingual BERT models. | en_US |
dc.rights | This article is distributed under the terms of the CC-BY-SA Licence. Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. | |
dc.subject | cs.CL | en_US |
dc.subject | cs.CL | en_US |
dc.title | Evaluation of contextual embeddings on less-resourced languages | en_US |
dc.type | Article | en_US |
dc.rights.holder | © 2021, The Author(s) | |
pubs.author-url | http://arxiv.org/abs/2107.10614v1 | en_US |
pubs.notes | Not known | en_US |
rioxxterms.funder | Default funder | en_US |
rioxxterms.identifier.project | Default project | en_US |
qmul.funder | EMBEDDIA: Cross-Lingual Embeddings for Less-Represented Languages in European News Media::European Commission | en_US |
qmul.funder | EMBEDDIA: Cross-Lingual Embeddings for Less-Represented Languages in European News Media::European Commission | en_US |