The 8 most used standards and metrics for Translation Quality Evaluation


2020-05-01 22:16 TAUS


阅读模式 切换至中文

The 8 most used standards and metrics for Translation Quality Evaluation In this blog post we will highlight some of the standards and metrics used in translation quality management. Why standards and metrics for objective evaluation? Different companies use different metrics which makes it hard to compare vendors, translators, projects and to benchmark translation quality with industry averages. In order to benchmark quality and productivity of translation services, we need an objective approach by employing industry standards and metrics. The difference between metric and standard is simple: a metric is a system of measurement; a standard is a required or agreed level of quality or attainment. A metric helps ensure that a service or a product complies with an agreed level of quality, the standard. In what follows, we will highlight some of the standards and metrics used in translation quality management. Standards ISO 17100 provides requirements for the core processes, resources, and other aspects necessary for the delivery of a quality translation service that meets applicable specifications. The use of raw output from machine translation plus post-editing is outside the scope of this standard. The ISO 9000 family addresses various aspects of quality management and contains some of ISO’s best known standards. The standards provide guidance and tools for companies and organizations who want to ensure that their products and services consistently meet customer’s requirements, and that quality is consistently improved. The EN 15038 quality standard is developed especially for translation services providers and aims to unify the terminology used in the translation field, define basic requirements for LSPs and create a framework for the interaction of customers and service providers in terms of their rights and obligations. A strong focus is on administrative, documentation, review and revision processes, as well as on the functions of different specialists who are involved in translation process over its duration. As a minimum requirement under EN15038 certification translations must involve at least two separate people performing translation and editing (or review). The ASTM F2575-14 is a standard guide for Quality Assurance in translation. It provides a framework for customers and LSPs desirous of agreeing on the specific requirements of a translation project. It does not provide specific criteria for translation or project quality, as these requirements may be highly individual, but states parameters that should be considered before beginning a translation project. As the document's name suggests, it is a guideline, informing stakeholders about what basic quality requirements are in need of compliance, rather than a prescriptive set of detail instructions for the translator. Metrics The LISA QA metric was initially designed to promote the best translation and localization methods for the software and hardware industries. While since 2011 LISA is no longer active, their standardization methods are still widely used in translation quality evaluation. This metric features three severity levels, but no weighting. The Model consists of a set of 20, 25 or 123 error categories, depending on how they are counted. The SAE J2450 metric has gained popularity in the manufacturing industry. It consists of four parts: Seven primary error categories which cover such areas as terminology, meaning, structure, spelling, punctuation, completeness, etc. Two subcategories: serious and minor Two meta-rules to help evaluators make a decision in case of ambiguity Numeric weights for each primary and subcategory The current version of the metric does not measure errors in style, making it unsuitable for evaluations of material in which style is important (e.g., owner's manuals or marketing literature). The newest metric for error-typology based evaluation is the MQM-DQF harmonized metric. LISA QA and SAE J2450 have not kept up with the times and lack the flexibility that is required in a world with much for diversified types of content. Because the need for an industry-wide quality metric was still great, both TAUS and DKFI decided to work on a new and better quality metric. The TAUS Dynamic Quality Framework (DQF) was developed in consultation with TAUS members. DQF includes various tools for the evaluation of translation quality, the error typology being one them. The Multidimensional Quality Metrics (MQM) is an error typology metric that was developed as part of the (EU-funded) QTLaunchPad project based on careful examination and extension of existing quality models. Despite the variety of approaches taken in industry and research, the two models turned out to be broadly similar, but they were also different in important ways due to their history. In a series of meetings the developers of MQM and DQF agreed to make substantive changes to both frameworks to bring them into harmony. The newly harmonized metric offers translation professionals a standard and dynamic model that can be used in every context. It can be used ‘stand-alone’ but is also available through the DQF open API. To conclude Without measurement, no improvement. Companies only become more efficient and deliver the required quality if they generate statistics and act upon the data. For that standards and metrics need to be implemented in translation workflows and technologies, streamline, automate and collect data in a normalized fashion. A number of CAT tools and TMSs have already implemented some of the metrics above. Most popular of these is the MQM-DQF harmonized metric integrated in the main CAT tools. For more information, please refer to this page:
本文将介绍8个在翻译质量管理中常用的规范和指标。 为什么要用规范和指标来进行客观评价? 不同的公司使用不同的指标,这使得比较供应商、翻译人员和项目,比较翻译质量与行业平均水平很难。为了衡量翻译服务的质量和效率,需要一个客观的方法,即采用统一的行业规范(standard)和指标(metric)。规范和指标之间的区别很简单:规范是要求或约定的质量水准;指标是一种测量体系。指标有助于确保服务或产品符合约定的质量水准,即规范。 在下面的内容中,我将重点介绍翻译质量管理中一些规范和指标。 规范 ISO 17100规范提出了翻译的核心流程、资源和其他方面的要求。这些都是提供符合规范的高质量翻译服务所必需的。机器翻译加译后编辑的输出模式不适用该规范。 ISO 9000系列涉及质量管理的各个方面,并包含一些ISO最著名的规范。这些规范为那些想要其产品和服务始终满足客户要求,质量得到持续改进的公司和组织提供了指导。 EN 15038质量规范是专门为翻译服务提供商制定的,旨在统一翻译领域中使用的术语,确定对语言服务提供商的基本要求,并为客户和提供商在权利和义务方面的互动建立一个框架。其强调的重点是行政、文件、检查和审校的过程,以及在整个翻译中参与翻译过程的不同专家的职能。作为EN15038认证的最低要求,必须有至少两个独立人员进行翻译和编辑(或审核)。 ASTM F2575-14是翻译质量保证的标准指南。该规范为希望就翻译项目的具体需求达成一致的客户和翻译服务供应商提供了一个框架。它没有提供翻译或项目质量的具体要求,因为这些要求可能是极个性化的,但列出了在开始翻译项目之前应考虑的参数。正如其文件名所示,它是一个指导方针,告知利益相关者需要遵守哪些基本的质量要求,而不是一套指导性的详细说明,供翻译人员使用。 指标 LISA QA指标最初是为软件和硬件行业推广最佳翻译和本地化方法而设计的。虽然2011年之后LISA不再活跃,但其标准化方法仍被广泛应用于翻译质量评估。该指标具有三个严重性级别,但没有权重。 该质量评估模型由20、25或123个错误的类型组成,具体取决于计数方式。 SAE J2450指标在制造业中得到了广泛的应用。它由以下四个部分组成: 七种主要的错误类型,包括术语、意义、结构、拼写、标点、完整度等方面。 两个子类:严重和轻微错误 两个元规则帮助评估者在出现歧义时做出决策 每个主要类别和子类别的数字权重 当前版本的指标不包括风格方面的错误,因此不适合用来评估要考虑风格的材料(例如,用户手册或营销文案)。 MQM-DQF是基于错误类型评估的最新指标。LISA QA和SAE J2450没有与时俱进,并且缺乏灵活性。全行业很需要统一的质量指标,因此TAUS和DKFI决定制定一个更好的质量指标。 TAUS动态质量框架(DQF)是在与TAUS成员沟通后制定的。DQF包括多种翻译质量评估方法,错误类型模型就是其中之一。多维质量指标(MQM)是一种错误类型指标。它基于对现有质量模型的检查和扩展,是欧盟资助的QTLaunchPad项目的一部分。 行业和研究采取的是两种大体相似的模型。但历史原因,它们在某些重要方面是不同的。在一系列会议中,MQM和DQF的开发人员同意对这两个模型的框架进行实质性的修改,以使它们基本一致。新的统一指标为翻译专业人员提供了一个标准和动态的模型。它的使用范围广泛,既可以单独使用,也可以通过DQF open API调用。 总结 没有评估,就没有改进。只有形成数据并根据数据采取行动,公司运作才能变得更有效率并满足质量要求。为此,需要在翻译的工作流程和技术层面使用规范和指标,以规范化的方式简化、自动化和收集数据。许多CAT工具和TMS平台已经融入了上述的一些衡量指标。其中最流行的是集成在主要CAT工具中的MQM-DQF指标。更多信息,请访问: