Machine Translation: Productivity as a Function of Quality

机器翻译:效率与质量的函数关系

2020-10-08 08:10 Nimdzi Insights

本文共442个字,阅读需5分钟

阅读模式 切换至中文

We recently introduced you to the two- (or five-) second rule, which is essentially the reaction or decision-making time a linguist should spend judging whether to post-edit a segment of machine translation (MT) output or to retranslate it. This rule of thumb aims to help increase the linguist’s productivity when working with MT. Another way of looking at the task of increasing productivity is through an MT auto-select feature we described here last year. It’s an approach that’s available in tools such as Memsource Translate, which also includes a Machine Translation Quality Estimation (MTQE) feature. MTQE helps users evaluate the quality of the MT output: scores are automatically calculated before any post-editing (PE) is done and appear at the segment level together with other translation resources (e.g., the translation memory).  These are the four MTQE scoring categories: MTQE could help limit or even eliminate the two-second MTPE rule. Using a built-in feature like this means linguists no longer need to make a decision whether or not to post-edit: the machine does it for them. Whenever a score of 75% or above is predicted by MTQE, the corresponding segment would be a candidate to start post-editing right away. When no score is predicted, the MT output can be discarded.  So what exactly does all this tell us about how MT quality and post-editor productivity are correlated? Let’s have a look at the chart below that shows how MTPE productivity (in words/hours) changes with increasing quality of MT. Source: Memsource. Productivity for EN>DE On the X axis, 100 means “perfect” MT: no post-editing needed. Productivity is plotted against productivity when translating from scratch (the flat lines). There are two productivity lines. They differ in the corresponding segment length (3 to 8 words, 9 to 26 words). The absolute productivity numbers may be a bit higher. But what’s important here is the observed trend: post-editing of low-quality MT is less productive than translating from scratch, and higher-quality MT increases productivity considerably. As noted in a LocWorld39 presentation, this is how Memsource measures performance (conceptually, the method is still the same): Source: Memsource. LocWorld 39, “Quality Estimation in the AI Era” At the far right are perfect MT outputs. And MTQE identified most of them as perfect (green bar), some as 75 (good MT, orange bar), and a very small amount as bad (blue bar).  This experiment proves that the post-editor’s productivity is a function of the MT output quality. MTQE is an example of how automated quality estimation can be incorporated and leveraged in the localization workflow to benefit productivity.
我们最近向您介绍了二(或五)秒规则,它本质上是语言学家应该花费的反应或决策时间,用于判断是对机器翻译(MT)输出结果进行译后编辑还是重译。 语言学家在使用MT时根据这条经验法则可以提高工作效率。 另一种研究提高效率任务的方式是利用我们去年在这里描述的MT自动选择特性。这种方法在Memsource Translate等工具中可用,Memsource Translate也包含有机器翻译质量评估(MTQE)特性。MTQE帮助用户评估MT输出的质量:在任何译后编辑(PE)完成之前分数都会自动计算,并与其他翻译资源(例如翻译记忆)一起出现在片段级别。 以下是四个MTQE评分类别: MTQE可以减少甚至消除MTPE的两秒规则。使用这样的内置功能意味着语言学家不再需要做出是否进行译后编辑的决定:机器会帮他们完成。每当MTQE预测的分数达到75%或以上时,可以用相应的片段将立即开始译后编辑,而没有预测的分数时,就可以丢回MT重新输出。 那么,关于MT质量和译后编辑效率之间的关系,我们从中究竟又知道了什么呢?让我们看看下面的图表,它显示了MTPE生产率(以每小时多少字计算)是如何随着MT质量的提高而变化的。 资料来源:MemSource。生产率EN>DE 在X轴上,100表示“完美”MT:无需译后编辑。从头开始翻译时生产力是平行于生产效率绘制下的直线。这是两条生产力线。它们在相应的段长度上有所不同(3至8个单词,9至26个单词)。绝对生产率数字可能要高一点。但这里重要的是观察到的趋势:低质量的翻译做译后编辑比从头开始翻译的效率低,而高质量的翻译显著提高了译后编辑效率。 正如在一个LocWorld39演示文稿中所指出的,这就是Memsource测量性能的方式(从概念上来说,方法仍然是一样的): 资料来源:MemSource。LocWorld 39,“AI时代的质量评估” 最右边是完美的MT输出。并且MTQE将其中大部分鉴定为完美(绿色条),一些鉴定为75分及以上(良好MT,橙色条),极少量鉴定为糟糕(蓝色条)。 实验证明,译后编辑的生产率是机器翻译输出质量的函数。MTQE就是这样一个例子,如何在本地化工作流中结合和利用自动化质量评估以提高生产率。

以上中文文本为机器翻译,存在不同程度偏差和错误,请理解并参考英文原文阅读。

阅读原文