Evaluating Machine Translation Quality Using Short Segments Annotations

Evaluating Machine Translation Quality Using Short Segments Annotations

We propose a manual evaluation method for machine translation (MT), in which annotators rank only translations of short segments instead of whole sentences. This results in an easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using thi...

Full description

Bibliographic Details
Main Authors: Macháček Matouš, Bojar Ondřej
Format: Article
Language:English
Published: Sciendo 2015-04-01
Series:Prague Bulletin of Mathematical Linguistics
Online Access:https://doi.org/10.1515/pralin-2015-0005
id doaj-8410fb61b1664506801d3ee4badd35df
record_format Article
spelling doaj-8410fb61b1664506801d3ee4badd35df2021-09-05T13:59:53ZengSciendoPrague Bulletin of Mathematical Linguistics 1804-04622015-04-0110318511010.1515/pralin-2015-0005pralin-2015-0005Evaluating Machine Translation Quality Using Short Segments AnnotationsMacháček Matouš0Bojar Ondřej1Charles University in Prague, Faculty of Mathematics and Physics, Institute of Formal and Applied LinguisticsCharles University in Prague, Faculty of Mathematics and Physics, Institute of Formal and Applied LinguisticsWe propose a manual evaluation method for machine translation (MT), in which annotators rank only translations of short segments instead of whole sentences. This results in an easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using this method. The obtained results are very close to the official WMT14 evaluation results. We also use the collected database of annotations to automatically evaluate new, unseen systems and to tune parameters of a statistical machine translation system. The evaluation of unseen systems, however, does not work and we analyze the reasonshttps://doi.org/10.1515/pralin-2015-0005
collection DOAJ
language English
format Article
sources DOAJ
author Macháček Matouš
Bojar Ondřej
spellingShingle Macháček Matouš
Bojar Ondřej
Evaluating Machine Translation Quality Using Short Segments Annotations
Prague Bulletin of Mathematical Linguistics
author_facet Macháček Matouš
Bojar Ondřej
author_sort Macháček Matouš
title Evaluating Machine Translation Quality Using Short Segments Annotations
title_short Evaluating Machine Translation Quality Using Short Segments Annotations
title_full Evaluating Machine Translation Quality Using Short Segments Annotations
title_fullStr Evaluating Machine Translation Quality Using Short Segments Annotations
title_full_unstemmed Evaluating Machine Translation Quality Using Short Segments Annotations
title_sort evaluating machine translation quality using short segments annotations
publisher Sciendo
series Prague Bulletin of Mathematical Linguistics
issn 1804-0462
publishDate 2015-04-01
description We propose a manual evaluation method for machine translation (MT), in which annotators rank only translations of short segments instead of whole sentences. This results in an easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using this method. The obtained results are very close to the official WMT14 evaluation results. We also use the collected database of annotations to automatically evaluate new, unseen systems and to tune parameters of a statistical machine translation system. The evaluation of unseen systems, however, does not work and we analyze the reasons
url https://doi.org/10.1515/pralin-2015-0005
work_keys_str_mv AT machacekmatous evaluatingmachinetranslationqualityusingshortsegmentsannotations
AT bojarondrej evaluatingmachinetranslationqualityusingshortsegmentsannotations
_version_ 1717812828992176128

Similar Items