Ethical Responsibility towards Underrepresented Languages

对任职人数偏低的语文的道德责任

2021-01-05 23:40 TAUS

本文共799个字,阅读需8分钟

阅读模式 切换至中文

"The Web does not just connect machines, it connects people," said Tim Berners-Lee, the inventor of the World Wide Web. Whether online or offline, language is just as important to building human connections: it forms the basis of how users identify with each other and the boundaries within which communities come together for common interests. Making information equally available in as many languages as possible online is strictly tied with the development of Machine Learning technologies and Machine Translation practices. Despite the efforts, not all of them gain the same traction or show the same results. This is shown by the research from academics Mark Graham and Matthew Zook, who compared the Google searches made in the West Bank in Hebrew, Arabic and English. They discovered a striking imbalance between linguistic groups: searches in Arabic in areas under Palestinian control usually result in only 5% to 15% of the number of results that the same search term brings up in Hebrew. English searches also bring back between four and five times more results than in Arabic. Inequality in information and representation in different languages online can even affect how we understand places and even how we act in these places. In a case study of the West Bank, searching for “restaurant” locally in Hebrew, Arabic and English brings back different results for each language. Google can send Arabic speakers to one part of the city and Hebrew speakers to another when they are searching for the same thing. This pre-selection of information that people receive risks reinforcing social segregation in the city and shapes how people interact with each other and the world around them. These studies make it clear that the universe of information on the internet looks very different from one language to another and what it often comes down to is the availability of data in a specific language. To help low-resource languages to become available online, TAUS started the Human Language Project (HLP) where communities of underrepresented languages generate datasets to be used in Machine Learning projects. Abdülselam Yıldırım is one of the ambassadors for the HLP Middle Eastern community. He speaks Kurdish, Kurmanji, Sorani, Zazaki and Hewrami in addition to Turkish, Persian and Arabic. He is the author of a Sorani-Turkish dictionary and several course books on Kurdish for Foreigners. He has been a well-respected linguist for 12 years and has translated 6 books from Persian and Turkish into Kurdish and Sorani. Abdülselam highlights that as a translator of several low-resource languages spoken by people in conflict and oppression zones, he understands the value and necessity of data production to support these communities and languages. “It is vital for me and the communities I represent to see that these languages that are not classified as official languages in countries they are spoken in, and are even banned in daily usage, are being valued by international companies and platforms,” says Abdülselam. He adds “The fact that the language data produced in these languages will be used in digital platforms will certainly expedite the connection and communication between oppressed communities and other cultures.” So far he has mobilized a big community of Kurmanji (Arabic and Latin script), Sorani and Turkish speakers to translate one million words in each one of these languages on the HLP Platform. These datasets are available for sale in the TAUS Data Library. Recently Abdülselam and his community translated 200K words in the English - Kurmanji/Sorani language pair in the media and news domain for use in AI and ML projects. These are available for purchase on the Data Marketplace. hbspt.cta._relativeUrls=true;hbspt.cta.load(2734675, '93c72fc5-17f4-4e74-b9d2-0d33f09adebb', {}); “I believe that making communication possible through machine learning applications, and thus faster, forms a new economic ecosystem of its own. It seems without a doubt that other linguists like me would see the value in this new business stream and understand how it helps us stay aligned with the contemporary business and technology trends,” says Abdülselam. Data will continue to be at the heart of the translation and language business going forward. The content production rate increases along with the speed of technology and AI has to keep up with it through diverse solutions. Based on this, Abdülselam suggests that “we must be aware that the wave of transformation has already started and it is the way that linguists, especially in low-resource languages, can monetize their efforts by mobilizing the communities of native speakers around them to produce datasets. This is a financial benefit for their communities but, more than that, ethical responsibility to their culture and languages.” hbspt.cta._relativeUrls=true;hbspt.cta.load(2734675, '40ddb533-d3bc-44f3-a11d-29b8fb916247', {});
万维网的发明者Tim Berners-Lee说:“网络不仅连接机器,还连接人。”不管是在线还是离线,语言对于建立人与人之间的联系同样重要:它构成了用户如何相互认同的基础,也构成了社区为了共同利益而走到一起的边界。 在网上以尽可能多的语言提供平等的信息是与机器学习技术和机器翻译实践的发展紧密联系在一起的。尽管作出了努力,但并非所有这些努力都获得了同样的牵引力或显示出同样的结果。学者马克·格雷厄姆和马修·祖克的研究表明了这一点,他们比较了在西岸用希伯来语,阿拉伯语和英语进行的谷歌搜索。他们发现了语言群体之间的显著不平衡:在巴勒斯坦控制的地区用阿拉伯语进行搜索,得到的结果通常只有用希伯来文搜索的结果的5%到15%。英语搜索带来的结果也是阿拉伯语搜索的四到五倍。 在线上不同语言的信息和表示的不平等甚至可以影响我们如何理解地方,甚至影响我们如何在这些地方行动。在西岸的一个案例研究中,用希伯来语,阿拉伯语和英语在当地搜索“餐馆”,每种语言返回的结果都不同。谷歌可以将说阿拉伯语的人发送到城市的一个地方,而说希伯来语的人则发送到另一个地方,当他们在搜索同一事物时。这种对人们接收信息的预先选择,有可能加强城市中的社会隔离,并影响人们如何与他人以及他们周围的世界互动。 这些研究清楚地表明,互联网上的信息宇宙从一种语言到另一种语言看起来是非常不同的,它通常归结为一种特定语言的数据的可用性。为了帮助低资源语言在网上变得可用,TAUS启动了人类语言项目(HLP),在这个项目中,代表不足的语言社区生成用于机器学习项目的数据集。 Abdülselam Yilderim是HLP中东社区的大使之一。除了土耳其语,波斯语和阿拉伯语外,他还会说库尔德语,库尔曼吉语,索拉尼语,扎扎基语和赫拉米语。他是索拉尼语-土耳其语词典的作者,还为外国人编写了几本关于库尔德语的教程。12年来,他一直是一位备受尊敬的语言学家,将6本书从波斯语和土耳其语翻译成库尔德语和索拉尼语。 Abdülselam着重指出,作为冲突和压迫区人民所说的几种低资源语言的译者,他理解数据生产对支持这些社区和语言的价值和必要性。Abdülselam说:“对于我和我所代表的社区来说,重要的是要看到,这些在他们所使用的国家没有被列为官方语言,甚至在日常使用中被禁止的语言,正在得到国际公司和平台的重视。”他补充说:“这些语言产生的语言数据将被用于数字平台,这一事实肯定会加速受压迫社区和其他文化之间的联系和交流。” 到目前为止,他已经动员了一个由讲库尔曼吉语(阿拉伯语和拉丁语),索拉尼语和土耳其语的人组成的庞大社区,在HLP平台上用这些语言中的每一种翻译100万字。这些数据集可在TAUS数据库中出售。最近,Abdülselam和他的社区在媒体和新闻领域翻译了20万个英语-库尔曼吉语/索拉尼语对词,用于人工智能和ML项目。这些都可以在数据市场上购买。 hbspt.cta._relativeURLS=true;hbspt.cta.load(2734675,'93c72fc5-17f4-4e74-b9d2-0d33f09adebb',{}); “我相信,通过机器学习应用程序使沟通成为可能,并因此更快,形成了一个自己的新经济生态系统。Abdülselam说:“毫无疑问,像我这样的其他语言学家会看到这一新业务流的价值,并理解它如何帮助我们与当代商业和技术趋势保持一致。” 数据将继续是未来翻译和语言业务的核心。随着技术的发展,内容的生产速度也在提高,AI必须通过多样化的解决方案跟上。基于此,Abdülselam建议,“我们必须意识到变革的浪潮已经开始,这是语言学家,特别是低资源语言的语言学家可以通过动员周围母语者社区制作数据集来实现其努力的货币化的方式。这对他们的社区来说是一项经济利益,但更重要的是,对他们的文化和语言负有道德责任。“ hbspt.cta._relativeURLS=true;hbspt.cta.load(2734675,'40DDB533-D3BC-44F3-A11D-29B8FB916247',{});

以上中文文本为机器翻译,存在不同程度偏差和错误,请理解并参考英文原文阅读。

阅读原文