Findings of the WMT 2023 Shared Task on Quality Estimation
Blain, F.
;
Zerva, C
; Rei , R.
;
Guerreiro, N.
; Kanojia, D.
; Sousa, J. G.
; Silva, B.
; Vaz, T.
; Jinxuan, Y.
; Azadi, F.
; Orasan, C.
;
Martins, A.
Findings of the WMT 2023 Shared Task on Quality Estimation, Proc Conference on Machine Translation WMT, Singapore, Singapore, Vol. , pp. - , December, 2023.
Digital Object Identifier:
Abstract
We report the results of the WMT 2023 shared task on Quality Estimation, in which the challenge is to predict the quality of the output of neural machine translation systems at the word and sentence levels, without access to reference translations. This edition introduces a few novel aspects and extensions that aim to enable more fine-grained, and explainable quality estimation approaches. We introduce an updated quality annotation scheme using Multidimensional Quality Metrics to obtain sentence-and word-level quality scores for three language pairs. We also extend the provided data to new language pairs: we specifically target low-resource languages and provide training, development and test data for English-Hindi, English-Tamil, English-Telegu and English-Gujarati as well as a zero-shot test-set for English-Farsi. Further, we introduce a novel fine-grained error prediction task aspiring to motivate research towards more detailed quality predictions.