Abstracts
Abstract
Recent developments in machine translation give hope for the possibility of communication without language barriers, as real-time interlingual conversations facilitated through automatic translation are already possible using free applications. This study aimed at measuring the level of intelligibility and accuracy of real-time chat messages translated instantly by translation bots embedded in GoogleTalk. The data consisted of chat scripts of a total of 12 sessions conducted between three pairs formed each of a native English- and Russian-speaker. The participants also answered a questionnaire about their chat experiences. The results suggest that even without any knowledge of the language of the other party, participants were able to conduct conversations on various topics without encountering any serious communication breakdown. About three fourth of the translated propositions were intelligible as well as accurate based on the human evaluation. Participants also reported positive comments on the effectiveness of this kind of interlingual communication, especially for informal chat.
Keywords:
- machine translation (MT),
- interlingual chat,
- real-time machine translation,
- machine translation evaluation
Résumé
Les développements récents dans le domaine de traduction automatique laissent entrevoir la possibilité imminente d’une communication sans aucune barrière linguistique, les fonctions de traduction automatique fournies par des logiciels gratuits permettant déjà à des locuteurs de différentes langues d’entrer en conversation en temps réel. La présente recherche vise à mesurer le niveau d’intelligibilité et d’exactitude grammaticale des messages de clavardage traduits instantanément par des robots de traduction intégrés au logiciel de messagerie instantanée de Google : GoogleTalk. Cette étude se fonde sur un corpus de messages échangés pendant 12 séances de clavardage en temps réel entre trois paires de participants, chacune formée d’un locuteur natif anglais et d’un locuteur natif russe n’ayant aucune connaissance de la langue de l’autre partie. L’analyse des messages échangés et des questionnaires remplis par les participants montre que les participants ont pu maintenir la conversation sur des thèmes divers sans rupture de communication majeure. Environ les trois quarts des messages se sont révélés à la fois intelligibles et exacts selon l’évaluation effectuée par des humains. Les participants aussi ont formulé des commentaires positifs sur l’efficacité de cette forme de communication interlinguale, en particulier dans le cadre d’une expérience de clavardage informel.
Mots-clés :
- traduction automatique,
- clavardage interlinguistique,
- traduction automatique en temps réel,
- évaluation de traduction automatique
Appendices
Bibliography
- Aiken, Milam, Rebman, Carl, Vanjani, Mahesh B., and Robbins, Tim (2002): Meetings without borders: A multilingual Web-based group support system. In: Proceedings of theAmerica’s Conference on Information Systems (AMCIS, 9-11 August 2002, Dallas, Texas).
- Aiken, Milam, Vanjani, Mahesh B. and Wong, Zachary (2006): Measuring the accuracy of Spanish to English translations. Issues in Information systems. 7(2):125-128.
- Aiken, Milam, Ghosh, Kaushik, Wee, John, et al. (2009): An evaluation of the accuracy of online translation systems. Communications of the IIMA. 9(4):67-84.
- Aiken, Milam and Ghosh, Kaushik (2009): Automatic translation in multilingual business meetings. Industrial management and data systems. 109(7):916-925.
- Aiken, Milam and Balan, Shilpa (2011): An analysis of Google Translate Accuracy. Translation journal. 16(2). Visited on 8 December 2011, http://translationjournal.net/journal/56google.htm.
- Calefato, Fabio, Lanubile, Filippo and Minervini, Pasquale (2010): Can real-time machine translation overcome language barriers in distributed requirements engineering?. In: Patrick Kellenberger, ed. 2010 IEEE International Conference on Global Software Engineering. (International Conference on Global Software Engineering, Princeton, 23-26 August 2010). Los Alamitos: IEEE Computer Society, 257-264.
- Ebel, Robert L. (1951) Estimation of the reliability of ratings. Psychometrika. 16:407-424.
- Goutte, Cyril, Cancedda, Nicola, Dymetman, Marc, et al., eds. (2009): Learning machine translation. Cambridge: MIT Press.
- Hutchins, John (2003): The commercial systems: The state of the art. In: Harold Somers, ed. Computers and translation: A translator’s guide. Amsterdam/Philadelphia: John Benjamins Publishing Company, 161-174.
- Hutchins, John (2005): Current commercial machine translation systems and computer-based translation tools: System types and their uses. International journal of translation. 17(1-2):5-38.
- Hutchins, John and Somers, Harold. L. (1992): An introduction to machine translation. London: Academic Press.
- Hutchins, John and Gaspari, Federico (2007): Ten years of online machine translation: origins, developments, current use and future prospects. In: Maegaard Bente, ed. MT Summit XI. (MT Summit XI, Copenhagen, 10-14 September 2007). Copenhagen: University of Copenhagen, 199-206.
- Kit, Chunyu and Wong, Tak Ming (2008): Comparative evaluation of online machine translation systems with legal texts. Law library journal. 100(2):199-322.
- Koehn, Phillip (2010): Statistical machine translation. Cambridge: Cambridge University Press.
- Nagao, Makoto, Tsujii, Jun-ichi and Nakamura, Jun-ichi (1985): The Japanese Government Project for Machine Translation. Computational linguistics. 11(2):91-110.
- O’Hagan, Minako and Ashworth, David (2002): Translation-mediated communication in a digital world: facing the challenges of globalization and localization. New York: Multilingual Matters.
- Solomon David J. (2004): The rating reliability calculator. BMC Medical Research Methodology. 4:11. Visited on 6 February 2012, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC411037/.
- Tatossian, Anaïs (2010): Vers une classification générale des variants graphiques des dialogues en ligne? Le cas du français, de l’anglais et de l’espagnol. Études de linguistique appliquée. 159:289-308.
- White, John S. (2000): Contemplating automatic MT evaluation. In: John S. White, ed. Envisioning machine translation in the information future:4th Conference of the Association for Machine Translation in the Americas, AMTA 2000. 1st ed. Pittsburgh: Springer, 100-109.
- Yang, Jin and Lange, Elke (2003): Going live on the Internet. In: Harold L. Somers, ed. Computers and translation: A translator’s guide. Amsterdam/Philadelphia: John Benjamins Publishing Company, 191-210.
- Yates, Sarah (2006): Scaling the tower of Babel Fish: An analysis of the machine translation of legal information. Law library journal. 98(3):481-491.