Vous êtes sur la nouvelle plateforme d’Érudit. Bonne visite! Retour à l’ancien site

The Visual Aspect of Translation Training in Multimodal Texts

  • George Damaskinidis

…plus d’informations

Note d’Érudit

Une modification a été apportée dans la bibliographie pour la référence de Damaskinidis, George (2015). Par conséquent, celle-ci diffère de la version imprimée (PDF).

Couverture de                Volume 61, numéro 2, août 2016, Meta

Corps de l’article

1. Introduction

It has been pointed out that the situation in higher education in a European (and only) context shows a general

lack of understanding of the true nature of translation teaching in university translation programmes, especially in terms of its purpose and the kind of end products it aims to have

Tan 2008: 253

Translation studies (TS) draw on several disciplines, both in theory and practice, such as linguistics, philology, semiotics, comparative literature, computer science, and terminology. This variety of disciplines has made difficult the formation of a single identity for translators. On the other hand, the formation of a focused training on the translation of the verbal has not been met with any resistance.

A field that has contributed most to remedy this imbalance is semiotics, defined by Saussure as ‘a science that studies the life of signs within society’ (Saussure 1916/1974: 16). Semiotics examines both linguistic and non-linguistic signs, where linguistic signs are represented by verbal elements (that is, written or spoken words) and non-verbal ones (for example, photographs, graphs, drawings).

Premised on Moriarty’s (2002: 26) claim that “semiotics provides a useful theoretical foundation to apply to visual communication because it helps unlock the complexities of visual communication,” here I examine if it is possible to apply semiotics to translation from a multimodal perspective. Thus, like Sütiste and Torop (2007: 203), who argue that “one and the same verbal text may exist within culture simultaneously as a verbal, multimedial, audiovisual, or audial text,” I start from the premise that it would be impossible to ignore the relationship between them. Yet, although the interaction of the visual and the verbal have blurred the boundaries of translation processes and semiotics, translation schools keep on emphasizing the verbal component of translation (Torresi 2008).

In this article, I review the literature that examines ways in which the visual semiotic mode in multimodal texts could be read and interpreted for translation purposes and the consequences that this has for the training in the translation of multimodal texts (Pérez González 2014). This examination takes place from the perspective of multimodal theories of communication (O’Halloran 2011) that have been proposed where meaning does not reside in language alone. Kress and Van Leeuwen (2006) argue that reading should bear in mind the semiotic resources that enable communication, the modes and the media employed and the communicative instance in which they are found.

Here, the term mode refers to the semiotic channel (for example, words, sounds, images, colour and animation) we use to compose a text, while the related term media refers to the tools and material resources (for instance, books, radio, TV) used to produce and disseminate (multimodal) texts (O’Halloran 2004; Kress and Van Leeuwen 2006). The multimodal text is defined here as a text whose “meanings are realized through more than one semiotic mode” (Kress and Van Leeuwen 2006: 183). The meaning of this multimodal text is created by the juxtaposition of a variety of semiotic elements (written, visual, aural) on the same interface (piece of paper or screen), as opposed to the monomodal one, such as a print verbal paragraph.

It should be stated that the article approaches multimodality from a mainly didactic perspective, rather than professionally. Translation teaching is the main point here and not the practices of professional translators. In addition, while my perspective to multimodality in this article is narrower than its full concept, the discussion of the visual mode on print documents could have relevance to other semiotic modes in electronic multimodal texts, such as movement, sound and gestures. Therefore, any reference to the ‘visual’ should be considered as only one of the many semiotic resources that constitute a multimodal text.

2. New challenges for translation studies

There is a widespread tendency towards the visual in Translation Studies (TS). The visual-verbal interaction is not a new problem for translators, but it requires constant re-interpretation. Also, the turn to the visual has given rise to new objects of study, where two relevant theoretical developments are social semiotics and multimodality.

It has been argued that literacy habits in the late twentieth and early twenty-first century have undergone dramatic changes. One of these is a tendency, among some educators, towards a more systematic approach to visual education, multimodality and literacy practices, where the dominance of the verbal modes of communication has been challenged by non-verbal modes. Recently, there has been an interest in problematizing the verbal/non-verbal divide in translation. In this article, verbal communication is considered to be any written or spoken word, while non-verbal communication consists of photographs, images, graphics, gestures, colour, etc. Neather (2008: 238) argues that “a far greater awareness of the ways in which differing verbal and visual imperatives shape translation is needed.”

The growing interest in multimodality as a theory of communication has given a new impetus to TS. In this context, the role of the non-verbal elements of the multimodal text to be translated has become a growing field of study attracting the interest of translation practitioners, teachers and researchers (Gottlieb 2005; Kussmaul 2005; Van Meerbergen 2009). In fact, most texts are multimodal rather than monomodal (for example, texts consisting only of written elements), in the sense that the written element is presented using a particular typeface or calligraphy.

Today, although translators are faced with the new challenges concerning the visual-verbal interaction, little work has been done in TS to meet them. These challenges include, according to Prieto Velasco, Tercedor Sánchez, et al. (2008: 6), “adapting or providing captions, transcripts and audio descriptions within the multimedia formats they work with.” In film studies, Baumgarten (2008: 8) explains, TS have de-emphasized “the role of the interplay between visual and verbal interaction in establishing the meaning of a film text, and how it may influence both the process of film translation and the finished product.” These authors suggest that there must be a way to discuss the meaning of non-verbal elements in relation to translation.

Van Meerbergen (2009) argues that a fuller examination of the multimodal source text (ST) and target text (TT) in translation requires an analysis of the interrelations between visual and verbal components. This kind of analysis calls for an integration of theories and methods from a variety of disciplines which are often considered to be outside the scope of interest of TS. Baumgarten (2008), in her analyses of the interaction of visual and verbal information in film texts and film translation, adopted an interdisciplinary approach to TS by integrating linguistics, visual analysis and cinematic narrative. Her acknowledgment that these disciplines at first sight seem disparate is corroborated by a widespread view that TS are not commonly combined with other disciplines.

Yet, it is not easy for translators to realize this when in principle, as Toressi (2008: 64) observes, “the term ‘translation’ is usually defined as a verbal-only practice.” Thus, it is a challenge to the traditional view of TS to expand the meaning of translation to cover non-verbal modes of expression. This difficulty is not unfounded since there are still questions such as: “Up to what point can we say that visual information in the image shapes the translation?” (Pettit 2007: 177). Also, she does not hesitate to argue that there may be ways in translation where one strategy could be preferred over another in the presence of non-verbal elements. Her argument seems to draw upon the notion of constrained translation, since its spatial-temporal limitations may influence translator’s decision-making when applying strategies for problem-solving. Although she specifically refers to subtitling, this approach will be shown to have applications in the translation of print multimodal texts. Subtitling is one area where visual analysis is dependent on the understanding of cultural and localization differences, as will be discussed later.

This type of multimodal research has started to be integrated into the discipline of translation (Kaltenbacher 2004). Linguists have begun to realize the various intersemiotic relations between certain verbal and non-verbal aspects, such as the translation of humour in film and comics. Just like “linguists have realized that purely monomodal discourse does not exist” (Kaltenbacher 2004: 194), so translators should realize that limiting the translation to the verbal component alone may miss a vital part of the message, or at least restrict the repertoire of semiotic elements available to the translator when attempting to translate the message. This view is also shared by Cosculluela (2003: 116), who states that TS “cannot be studied in a satisfactory manner from the point of view of linguistics alone.” Similarly, Risku and Pircher (2008) argue that graphic design is an issue to be taken into account in translation.

Before moving on, it is necessary to clarify the terms multimodal and multimedia. Some authors (for example, Lauer 2009; Remael 2001) have made clear the distinction between these terms by relating the former to a public/industry context and the latter to the academic field. The difference between multimodal and multimedia is mostly a difference between ‘modes’ and ‘media.’ Another way to distinguish them is to place the mode on the ‘content’ side and the media on the ‘expression’ side of the meaning-making process. Moreover, while mode could be seen as involving issues of design and process, media is about production and distribution.

In order to contextualize multimodal and multimedia in this research, it is also important to realize that “although media and modes are different from each other, the media we use affect the ways in which we can realize meaning through various modes” (Lauer 2009: 227). For example, the mode of writing in the medium of the book interacts with the writer and reader in a way that differs when the medium is the screen. If this is the case, it is not a coincidence that most proofreaders print the electronic texts before they start proofreading them. I argue that the reading of multimodal texts should be characterized by the translator’s effort to understand different modes in making meaning of texts.

3. Visual implications for translation trainers and students

Bearing in mind the discussion thus far, practitioners and trainers of translation are faced with a new relationship between translation and literacy, within the new field of ‘visual literacy.’ A very basic definition of visual literacy is “the ability to understand and produce visual messages” (Bleed 2005: 5). Thus, visual literacy is defined here as a group of competencies that a translator should develop in order to interpret visual messages when translating multimodal texts into another language.

From a translation pedagogy point of view, the visual and the verbal, according to Sütiste and Torop (2007), have also blurred the boundaries of translation processes and semiotics. It could be argued that multimodal translation is governed by semiotics, rather than the more narrowly defined operations of translation. Also, language faculties offering translation are not providing adequately if students have not been prepared in semiotics. This blurring highlights the interdisciplinary nature of training for would-be translators, which it is argued may entail new skills for the ‘multimodal semiotics translator,’ a term referring to the translator who adopts both a semiotic and multimodal approach to translation. According to Torresi,

a translator who is aware of the importance of non-verbal elements, and the resource they represent for translation, proves a more reliable team-worker and produces better target texts

Torresi 2008: 70

In more pragmatic terms, translators that gain more semiotic skills could persuade clients that translators are not just ‘word-mongers’ but text- and meaning-makers.

For similar pedagogical reasons, Tan (2008: 590) urges translation teachers to follow “a broader road of translation education.” However, this approach to education may not be easily integrated in TS since translation pedagogy has shown, at least in technical communication, a “strong focus on the verbal in translation training and practice […] ignor[ing] the use of images and other media” (Risku and Pircher 2008: 162). But for a few exceptions (for example, Monterde Rey 2005; Prieto Velasco 2008), technical translation, which is abundant in visual elements, has turned a blind eye to the training and practice of using non-verbal elements, in other words, to visual literacy.

In order to develop the necessary key concepts related to the role of the visual in the translation of multimodal texts there is a need for even more empirical studies (Van Meerbergen 2009). In fact, this issue will not be resolved from a purely theoretical perspective, but requires actually putting students into a relevant situation and exploring what happens. Here, I review relevant existing literature and describe ways of approaching the visual semiotic elements of multimodal texts in the translation classroom, mainly from a pedagogical viewpoint.

The change of focus in TS, from seeing the TT as a static product to looking at its process of translation, has been a major development in the training of translators. Innovative teaching projects (Tercedor Sánchez and Abadía Molina 2005; Tercedor Sánchez, Alarcón Navío, et al. 2009) involving multimodal text production and semiotic analysis of images have shed new light on translation education. Similarly, Torresi (2008) argues that translators should be ‘text-makers’ and able to deal with issues such as layout and typography in producing a TT in a multimodal format. The importance of understanding the role of multimodal texts in TS has also been raised by Van Meerbergen (2009), who claims that certain concepts such as ST and nationality can be problematic to deal with in producing the translation of multimodal texts. In the context of translating texts consisting of a high number of visual elements, the notion of creativity in relation to the visual in translation has gained new impetus (Cho 2006).

4. Translation training in multimodal contexts

4.1. A multimodal approach to translation training

It has been realized that one aspect of the problem is reflected in the emphasis placed on the verbal-based training usually provided by translation schools. Torresi (2008: 64) points out that “[translation] training tends to focus on the verbal dimensions of the text that are relevant for the science of linguistics, treating as incidental, if at all, any non-verbal elements.” Moreover, Risku and Pircher (2008) propose translation training courses that would make future translators more aware of the importance of non-verbal elements.

A new tendency in the training of translators, according to Sütiste and Torop (2007), is the introduction of intersemiotic translation, for reasons that are both pedagogical (comprehension of the visual aspect of the text) and pragmatic (translating into a visual environment, such as a newspaper layout, etc.). This type of translation pedagogy is also an indication of the changing boundaries of translation processes. If we accept Smith’s (2008) claim that translators should be involved in the creation and adaptation of global advertisements, then their training should include a basic knowledge of all the issues and procedures related to producing such multimodal texts; an area not directly related to translation.

In translation training, the term multimodal text/approach is “theoretically accurate to describe the cognitive and socially situated choices students are making in their [translations]” (Lauer, 2009: 225). Additionally, the term being more familiar within academia, and coupled with the fact that the texts used in this research with the students are print documents, it was considered more appropriate to use the term multimodal to avoid misunderstandings with the term multimedia, which is more associated with screen-based texts.

4.2. Developing a translation awareness of multimodal texts

Since Remael (2001) put forward the idea of teaching how to produce multimodal texts in TS, there have not been many examples of explicit training in multimodal text production for translation purposes. However, the examples that do exist are quite illustrative of the direction translation training could take.

Such an example is provided by Prieto Velasco, Tercedor Sánchez, et al. (2008) who gave their students technical and scientific multimodal texts to be translated. In this activity, they asked them to describe the visual material of the TT and produce informal think aloud protocols at home by recording their own descriptions. While data is not provided to evaluate the effectiveness of this teaching approach, these transcribed verbal reports produced by the students during the translation process could be taken as the first step towards formulating the TT. The results of this analysis, Kussmaul and Tirkkonen-Condit (1995: 178) argue, “can then form a basis for translation pedagogy.”

These new types of multimodal texts will find a place in the translation classroom only if we “widen our definition of writing to include multimodal composing as a newly available means” (Hull and Nelson 2005: 29). One way to widen this kind of understanding of multimodality and translation is to refer to the idea of ‘intergeneric translation’ (translating from one text genre into another) or ‘heterofunctional translation’ (translating a ST into a TT with a different function or skopos). The composition of multimodal texts, which involves a reception (reading a ST) and production (writing a TT) stage, could become a controlling-influence activity for the multimodal text translator. Schubert (2009), for example, argues that translators may be asked to produce a TT whose design (for example, especially technical documents) may differ from the ST’s. That is, the commissioner of a translation may ask, for some reason, a TT whose appearance would be clearly different from the ST. In the case of a technical text, the placement of graphics or drawings on a different position in the TT, with a different size, or in a different format (for example, from a pie graph to a circle or bar graph), may have an effect on translation, in terms of creating new reading paths, of the salience of graphs and drawings and of the new text that accompanies their different formats. It is reasonable to assume that translation training (at least in technical translation) should include an aspect of producing whole texts, rather than dealing with the verbal only material.

This approach to translation training would greatly enhance translators’ ability to deal with these new requirements so as to meet the customer’s needs. The idea of simultaneously handling visual and verbal material could also contribute to Gottlieb’s (2005) intention for a multidimensional approach to translation where no semiotic mode is undermined at the expense of the others. For instance, the delay of the translation process so as to deal with the non-translation task of re-designing according to the TT conventions (or to take into account the new design) might change the textual relationship between the verbal and the visual text components (Van Meerbergen 2009).

The combined task of re-designing and translating is one of the occasions where ‘translation goes beyond the mere reproduction of the ST in another language’ (Schrijver, Van Vaerenbergh, et al. 2011: 3). If re-design is considered to be a type of editing performed on the (non-verbal part of the) multimodal text, then the concept transediting (Stetting 1989), a composite term of translation and editing, is relevant to the production of technical (and thus multimodal) texts in a translation context. Stetting (1989) refers to transediting as an adaption of the ST, in terms of the language, the intended function and the culture of the TT. Chesterman, cited in Schrijver, Van Vaerenbergh, et al. (2011: 2), describes transediting as “the sometimes radical re-editing that translators have to do on badly written original texts: it includes drastic re-ordering [and] rewriting.” Stetting (1989: 379) shares this markedly new approach by presenting “transediting on a spectrum ‘at one end [the free end] followed by dynamic translation’.”

Despite these new requirements imposed on translators, multimodal text production techniques in translation have been neglected. For instance, Schrijver, Van Vaerenbergh, et al. (2011) argue that rewriting – another term for transediting – especially of poorly composed STs, is a professional reality for translators, which has not been examined extensively in translation-process studies. Torresi (2008) makes a similar point by arguing that most of the didactic practices in mainstream translation classrooms are ‘verbocentric,’ with very few exceptions, such as in multimedia translation. Some of the reasons for this disinterest could be attributed to some very practical and sensible reasons; for example, verbal-only training provides students and teachers with a clear focus on the verbal dimension of the text without devoting time to the non-verbal dimension that complicates things. Multimodal text production teaching and learning is time-consuming, and talking about pictures adds considerable time pressure to classroom time. Finally, it may distract students from the primary skills required in the translation market.

Tercedor Sánchez, López Rodríguez, et al. (2005) propose that translation teachers should move away from these text-centred approaches in the translation course and embrace those that would include images. In technical and scientific web documents, there are images accompanying a text that do not have a description, and this poses problems in their retrieval by search engines. In such cases, they asked translation students to propose a text description (in the ‘alt’ attribute of an html document) for these images even if there is no ST verbal element. Moreover, even in cases where such a description does exist, they go on, in the TT it may have to be modified because the same description is given for different pictures in the site. This technique could be developed even further and introduced in the translation of print multimodal texts containing images so complicated as to be incomprehensible to the TT readers.

In the context of teaching multimedia translation, Tercedor Sánchez and Abadía Molina (2005) had their students produce text descriptions (of images) in the TT when they were absent in the ST. The students were given pictures devoid of text (image-to-text activity) and were asked to describe them denotatively and connotatively. They were then asked to write down the lexis evoked by images. Finally, they had to suggest an appropriate context for these images to appear in. As a text-to-image activity, the students were given verbal elements and were then asked to visualize an appropriate picture. By visualizing a concept, the students had less difficulty in finding an appropriate translation solution.

The identification and description of visual material is a difficult task, and in some cases culture-dependent. Forceville (1996) conducted an experiment where he asked a (Chinese) respondent to identify and describe an advertisement for IBM where the only verbal element was the IBM logo. The problems faced by this person suggest, according to Forceville (1996), that pictures may not cross borders more easily than words and their interpretations may be dependent on cultural background knowledge. Therefore, it is proposed that any similar teaching activity, at least for translation purposes, should be carefully designed to avoid confounding cultural factors.

Setting up exercises on the translation of the same multimodal text with and without its visual elements would prepare translation trainees to deal with real life problems. Torresi (2008) argues that it is not uncommon for clients to give translators a text without some or all of its visual elements. These elements may include a missing photograph, tabular information out of its table, a paragraph out of its box and other graphic conventions that will be adopted in the final version. If translators are denied access to this type of information they will also be left out of comprehensive text-making processes and subsequently fail to take into account all the modes in which the TT is to be encoded. These processes may also include the omission of certain non-verbal elements in the TT.

Horn-Helf (2005) claims that translators may have to modify technical documents in various ‘creative’ ways (see also section 4.3); for example, by omitting certain visual elements (such as dimensional drawings), changing the types of figures (from pictures to sketches) or modifying the way they are placed on the page (from left-hand side view to right-hand view) in multilingual brochures to satisfy cultural conventions, or for localization/domestication purposes. While localization will not be discussed in detail here, its relevance lies in the fact that it

is a kind of intersemiotic or ‘crossmedium translation’ as it is a transfer that involves more than one language, but also more than one medium, that has become a fast-increasing social and economic reality

Valdés 2008: 227-228

Valdés (2008) argues that localization is important for translators in order to communicate successfully to different markets and cultures that involve the translation of texts with multiple semiotic resources. In the case where the medium does not change, as the print advertisement, the need for intercultural communication may require translators to find images more familiar to the TT culture to accompany the verbal part of the advertisement. This requirement would depend on an in-depth study of the target market and culture. Besides finding alternative images, the localization process may also involve omitting visuals in the TT that exist in the ST or adding visuals in the TT that do not exist in the ST.

Horn-Helf (2007) describes how in the Russian translation of a German technical book, all photographs and four engineering drawings were omitted, while one chart was added. She speculated that these omissions could be attributed to a USSR-era practice where it was prohibited to take pictures of industrial installations or to reveal sectional drawings of process equipment. In these ideologically driven translations, we can only imagine the difficulties the translators must have faced when they were asked to follow these socio-cultural conventions and make the required omissions. Therefore, it seems a good teaching technique to introduce students to the translation of language pairs as diverse as English and Russian, at least in technical translation.

From a more radical perspective, Torresi (2008) openly declares that the verbal-only approach to translation training should be abandoned altogether. In order to highlight this need, Torresi (2008) gives the example of graphic designers possibly omitting a final paragraph without consulting the translator because ‘it didn’t fit into the space’ of the PDF file. In other cases, translators may dismiss as irrelevant the graphic designer’s warning to keep the TT the same length as the ST. Even though such dogmatic positions are not easily adopted, in translation training involving some types of texts, such as advertisements, these strategies may work.

Advertising translation is one of the fields in TS where the traditional linguistic quality assessment criteria, such as faithfulness and respect for the ST, disappear altogether. Torresi (2008) argues that the success or failure of the translation of an advertisement lies in its commercial outcome. Since advertisements are mainly (if not almost always) multimodal texts, there is a need to establish relevant quality assessment criteria. Yet, there is the paradox that while the verbal part of the translation of the multimodal text will be assessed against the traditional linguistic quality assessment criteria, the translation of the multimodal text, as a complete TT, will be assessed against some other (unknown thus far) criteria. Therefore, it is important to develop these criteria to be taken into account by translators so that they can have a say in the production phase of the TT as a whole and not only in the translation of the verbal elements.

Smith (2008) points out that, based on their working experience with illustrations and issues of layout, translators should be consulted on the appropriateness of visual configurations. While this may be true to an extent, here translators are called upon to take part in two different processes: firstly, in the translation of multimodal texts, where their responsibility is to translate the verbal elements, and secondly, in the entire process of producing the multimodal TT, from start to finished product, which entails working as a team with the other stakeholders in the project, such as the graphic designer, the typesetter or the typographer.

This procedure would require a type of training that Kiraly (2001) calls a social constructivist and collaborative learning approach to translation training pedagogy. The importance of collaboration among students as a multiplier of multimodal-text creation has also been highlighted by Mills (2010). This collaborative work is seen as a strategy diametrically opposite to the dominant discourse of teacher monologue. This line of thinking would require translation teachers to adopt a reflexive approach to their own practices by allowing students to evaluate the teaching and learning processes in the classroom.

According to Tercedor Sánchez, López Rodríguez, et al.’s (2009) evaluation of one of their own translation courses involving images, their students became aware of two fundamental aspects regarding images: their key function in texts, and the type of information transmitted by them. In a previous evaluation of a similar course, Tercedor Sánchez, López Rodríguez, et al. (2005: 145) claim that, in translation, images should be described “with regard to their functional role within the text, for the purpose of interpreting their connotative features.” Especially in technical texts, translators need to infer relevant aspects of visual concepts (for example, graphs, diagrams, tables, maps) to develop visualization strategies with regard to them. This approach to analyzing images facilitates the learning of domain-specific terminology.

These transformations require would-be translators to become literate in multimedia tools, visual design, photo-editing techniques, etc. If translation teachers lack competence in specific areas, or if they have to become semi-specialists in several subject fields, then consideration must be given to their professional development. An efficient way towards this direction is to foster visual creativity in translation students.

4.3. Using visual stimuli for fostering creative translations

In the context of translating texts consisting of a high number of visual elements, the notion of creativity has gained new impetus. Kussmaul (2005: 379-380) defines creative translation as “visualizations [that] lead to shifts, transpositions, modulations etc., in other words, the translation involves changes when compared with the source text, thereby bringing in something that is novel.” It is this addition in the TT, absent in the ST, that makes the translation a creative one.

Since images are not constrained by the traditional rules of verbal language, “creative translation [may be] related to translations with [an] unpredictable, non-institutionalized use of language” (Cho 2006: 3). In the translation of multimodal texts, if we accept that non-verbal elements could also assume the role of the verbal, then an extended view of language may also involve the use of visual material. Since translation cannot be totally creative, in the sense that it is not created out of nothing but governed by the ST, visualization and creativity could be approached as an instance of rewriting elements of the ST.

Based on the hypothesis that creative thought is visual thought, Kussmaul (2005) proposes the investigation of four visual stimuli: looking at real pictures, at suggestive frames in the ST, at suggestive scenic details in the ST or imagining scenic details constructed from memory. This investigation is based on the frame and scene theory, which is intended to facilitate the visualization process for creative translations. Kussmaul (2005) applies this theory in TS and claims that by visualizing a scene (mental picture) fitting a word (frame), the translator accomplishes a creative translation.

In a small experiment with his students involving the translation of the DVD cover of a film, Kussmaul (2005) found that although the students had not seen the film, the photograph of a scene on the cover helped them to visualize and produce a creative translation. This creativity had to do with a change of focus, from the participants (ST) to their specific situation (TT). Such an application of the visualisation and frame/scene theory in translating, according to Cho (2006), increases the chances of creative translations. It is proposed that forms of translation creativity other than literary and poetic texts be explored, namely more practical ones. This tendency is highlighted by Torresi (2008) who argues that, in marketing, visualization may also represent a first stage of intersemiotic translation. It is suggested that a creative translation of multimodal texts into a given culture should include the manipulation of the visuals in order to make it more appealing for commercial purposes.

Based on the same hypothesis that the process of visualizing and describing images from different perspectives triggers creativity, Tercedor Sánchez, López Rodríguez, et al. (2009) had their students produce texts by considering image-text relations from two opposite directions: ‘from-text-to-image’ and ‘from-image-to-text.’ In the second direction, the students were asked first to analyze images devoid of verbal material, and then to make a list of the lexis evoked by these images. Before beginning their translations of the original ST, the students had to choose an appropriate context for these images, which included identifying the person who commissioned the work, potential recipients, and the primary use of the images. This further suggests that translation involving images is an activity that fosters creativity.

Tercedor Sánchez, López Rodríguez, et al. (2009) conducted another series of experimental studies to demonstrate how the visualization and description of images could trigger creativity in translation. They propose two types of activities for translation courses: “analysis and description of images,” and “strategies that link the visual and verbal components of the text with previous and newly acquired knowledge” (Tercedor Sánchez, López Rodríguez, et al. 2009: 165). Translation problems involving the use of images in multimodal texts were solved by adopting creative solutions, such as image-based documentation and the production of other multimodal documents that were illustrated by these images. Although the experimental studies focused on technical translation and localization, these two activities could be adapted to any course, since images could appear in any type of text. Also, although technical translation, according to Byrne (2006), may not be creative but simply a reproductive transfer process, the production of technical multimodal texts boosted students’ creativity and connected translation tasks with technical writing.

Cho (2006) and Kussmaul (1995) point out that creativity (not only in translation) is not a talent but a skill to be acquired through training and education. Cho claims that if translation can be taught and further developed through knowledge of, and practice in, at least two languages, so can creativity. This is in agreement with Kussmaul’s claim that no one is gifted with creativity but that it is a basic feature of mind, and as such, anyone can be creative in translation.

Similarly, Niska (1998) highlights the uniqueness of each individual in novel production by relating novelty to creativity. The term ‘novel production’ refers to any attempt by the translator to resort to “novel ways of encoding an old message” (Neubert 1997: 19). Such novelty is usually observed in translation solutions that deviate significantly from any precise and empirical definition found in a lexicon. While Niska (1998) does not dismiss the idea of having translators trained in creative methods of translation, he points to the individual’s inherent capacity to be open to new experiences, to form his/her own basis of evaluation and to experiment with new elements and concepts. What follows is that novelty, at least in the translation of multimodal texts, is a type of creativity. For example, in order to produce a novel word (for example, a neologism), the translator would have to be creative, while a creative translation may not lead to a novel production, but simply to a successful choice. Therefore, it is important to understand this distinction when analyzing translations that involve visual elements so as to develop a more refined understanding of the translation of multimodal texts.

Though the authors in the previous paragraph may be right in principle, further investigation is required before we argue that the acquisition of creativity and translation share common characteristics and requirements. It is probably for this reason that Kussmaul (1995) is cautious about the prospects of education in translation creativity, describing such an endeavour as an ‘ambitious aim.’ However, the concept of creativity may be particularly relevant to the training of multimodal text translation because the various non-verbal elements allow for deviations from proper translation, such as the examples above in subtitling and museum panels. The inherent difficulty in talking about images, which may involve using technical photographic terms, without a supporting text raises the question of what kind of knowledge or special skills students should have acquired in order to ask specific questions about an image prior to translating. These new perspectives reveal the concern surrounding the need to consider more widely a range of approaches to translation training, such as the employment of subject specialists in the translation classroom.

4.4. Bringing subject specialists into the translation classroom

Multimodal texts, especially those with complex layouts and technical content, pose a challenge to translators because the latter would normally not have the necessary expertise to deal with these issues. It is high likely that they would have to consult other experts and work in a collaborative manner. The idea of translators working closely together with other stakeholders, such as fellow translators and researchers, teachers, students (when training is in the form of teamwork) and clients, is not a new one (Oittinen 2008). According to Torresi (2008: 69), “the translation of complex multimodal texts could be the product of a long decision-making process, not carried out by translators alone.” These processes involve the collaboration with a number of professionals from other fields.

Torresi (2008) argues that translators may have to consult with the person who commissioned the translation and with professionals in specific fields. In advertising translations, which involve the adaptation of brand images, the translator’s work is only one of a series of steps in the translation process, beginning with the commissioning of the translation and ending with the TT as a final product. This approach not only takes translators away from translation proper; it also places them in a professional environment which they have not been prepared for. This trend is highlighted by Van Meerbergen (2009), who discusses the way in which international co-productions of Swedish translations of Dutch picture books have led publishers, editors, writers and translators to work closely together.

This list could grow by adding experts in various subject fields. Schubert (2009) calls experts ‘informants’ who are mainly called upon to offer their expertise in technical information. Such a contact, according to Schubert (2009), is an activity that takes place mainly at the ‘information research’ stage. However, the translator could be in contact with an expert not only in the pre-translation stages, but also while translating a text and in the post-translation stage, when revising the translation. Seeking expert help to gain a better understanding of specialized texts is a habit that translation students should develop in the early stages of their training. Moreover, this habit could be developed into a skill, because it is one thing to ask for help and quite another matter to ask the right questions.

By developing the skill of identifying problems that cannot be solved by translators themselves, they actually “raise problems and stimulate the client or the professionals they work with in order to find solutions together in the quickest possible way” (Torresi 2008: 71). This does not mean that translators of multimodal texts and the experts consulted should ‘trespass’ on each other’s professional field of expertise, but rather that they should work in a collaborative way. An example of this kind of collaboration is provided by Walsh (2009), concerning a teacher who invited an expert to a film study classroom to demonstrate the iMovie application. In the context of the translation classroom, the teacher could have arranged for the iMovie manual to be translated and then have had the expert answer the students’ questions.

The call to bring experts into the classroom is also shared by Tercedor Sánchez and Abadía Molina (2005), who point out the importance of translators working in cooperation with specialists. Particularly in technical translation, metaphoric images can be deceiving to non-specialists and translators might find it difficult to identify the nuances of meaning. In medical translation teaching, Wakabayashi (1996) urges her students to contact the author of the translation task at hand for two reasons: firstly, to request copies of very recent references cited in the ST to gain background knowledge about the content of the text, and secondly, to obtain, among other things, information about non-verbal elements, such as colour photos for checking the exact colour to which reference may be made.

This teaching technique is also an instance of the increasing awareness of the role non-verbal elements may play in translation. Although Wakabayashi (1996) claims that, in a classroom scenario, obtaining this type of material (for example, colour photos) would be almost impossible, I would propose sending the students to contact the client in person and discuss any relevant issues. However, in this case, it might be appropriate to contact clients in advance so as to fit this meeting into their professional schedule. In this collaborative mode of learning, the translation teacher acts as facilitator in the learning process and the classroom is not separated from the real world. Prieto Velasco, Tercedor Sánchez, et al (2008) simulated a real-world condition for their students by arranging a translation brief with real clients and experts in subject fields. Though they do not give details about this activity, we could assume that it might include regular visits to the clients’ and experts’ workplaces. Additionally, seminars could be arranged with field experts in the disciplines in which the students are translating.

Translation students who have regular contact with subject experts will have acquired additional skills. Kussmaul and Tirkkonen-Condit (1995) argue that students who acquire expert knowledge in subject fields, coupled with traditional translation skills, will have an additional asset upon entering the translation market. In order to provide this type of training, translation teachers could play the double role of the ‘expert in a given field and an expert in [translation] teaching’ (González Davies 2004: 2). However, the call for translation teachers – and professionals as well – to be experts in a field should be seen in close relation with the reality that a good translator cannot specialize in a large number of subject areas (Byrne 2006). Even when teachers are armed with a good and solid understanding of the basic principles and technologies of a subject field, they will never reach the level of competence that would make the professional expert in the classroom redundant. Obviously, teachers, students and experts make up a challenging educational mixture in the translation classroom. In these ‘unconventional’ learning situations, the teacher’s catalytic role in this bidirectional mode of learning is, according to Kiraly (2001), an example of a social constructivist and collaborative learning approach to pedagogy.

There is however the risk, according to Kemmis (1988: 176), that the intervention of outside facilitators may “introduce significant distortions in [the] practical, collaborative, or self-reflective” nature of action research. This distortion may take the form of

the legitimation of practices by reference to outsiders’ reputations or ascribed status as “experts” or “authorities” rather than being based in the practical discourse of practitioners themselves

Kemmis, 1998: 177

In reference to the translation classroom, the presence of the specialist may cause an imbalance between expert advice and the time allocated for creative brainstorming with the students.

Bringing together students and subject specialists is an opportunity for the former to hone what Lee-Jahnke (2005) calls ‘extra-linguistic abilities,’ which include specialized knowledge of a specific domain. If we organize translation projects that would enable students to collaborate with these subject specialists, it would also be possible to achieve “maximum effect in autonomous learning and responsibilization” (Lee-Jahnke 2005: 362). This team spirit is highlighted by Risku and Pircher (2008), who propose that, at least in intercultural technical communication, translators and technical communicators could provide valuable support in the production of technical multimodal texts.

In such a complex translation project, translation students would also have to work with other stakeholders in product design, development and marketing (Gambier and Gottlieb 2001). Moreover, the students would have the chance to collaborate with the producers of non-verbal elements (for example, graphics, photographs, maps) and gain more in-depth knowledge of complex extra-linguistic material. Torresi (2008: 70) has also related teamwork with multimodal translation by arguing that the translator who is “aware of the importance of non-verbal elements, and the resource they represent for translation, proves a more reliable team-worker and produces better target texts.” Torresi refers to a translator who contacted a publishing studio to suggest a change in a visual (translating an English sign into Italian) by means of photo-editing software. Although the visual remained unaltered, a translation of the notice was added as a caption.

However, there is no strong theoretical or empirical evidence that could relate non-verbal element awareness directly to increased translation competence or even to the ability to work cooperatively. Various issues of translation training, such as whether the suggestions for curriculum development are realistic and how they could be fitted in, will be discussed in the next section.

5. Identifying visually-oriented teaching areas for further development

Recently, several innovative teaching practices have been adopted in translation training. Although this training is not labelled as multimodal training, it has all the characteristics of incorporating multimodal-based techniques. Translation trainers have started to realize that new pedagogical and pragmatic reasons dictate a change in translation training. Although most of this training is restricted to specific fields, such as scientific and technical translation, or advertising, it is indicative of the new trends in translation training. Unless stated otherwise, the following are pedagogical proposals to be adopted by translation teachers in multimodal training contexts.

The production of multimodal texts is proposed as a teaching technique per se or even a mere description of images without the translation of verbal elements. The second activity has already been implemented by Tercedor Sánchez, López Rodríguez, et al. (2009) and highlighted the importance of focusing on the information transmitted by images. Another exercise that is put forward is the omission or addition of visual elements (for example, photographs and images) in the TT, which is like producing a new multimodal text. This exercise may be dependent on collaborative work with other stakeholders (for example, graphic designers), on extensive reading of the relevant literature (for example, technical manuals) or on multi-cultural knowledge (for example, producing multi-language versions of manuals or illustrated picture books).

Specific fields might offer opportunities for collaborative learning. In advertising, for example, students from various disciplines (for example, translation, graphic design, advertising, business administration) could form a team and be assigned the task of rendering an advertisement/multimodal text in the language of another culture. In this team, all the members would have a specific job to do, by taking care to examine the impact of their work on the rest of the team. Another way to raise multimodal awareness is through the description of images even when they are totally devoid of an accompanying verbal element.

The relation of creativity to the visual aspects of translation is a relatively new concern in TS. This creativity is mostly associated with the production of multimodal texts, and in particular with the description of images. Even the fact that translators are looking at an image and trying to describe it might help them to arrive at a creative translation. Yet, there is the problem of defining whether (and to what extent) creativity is an inherent capability or whether it can be taught. Also, similarly to production techniques, examples of visual stimuli and creativity in translation have been found mostly in the context of teaching technical and scientific translation.

It is common belief that there will be times when the translator will seek an expert’s advice. Unlike multimodal text production techniques, the role of the subject specialist in the translation classroom is not so clearly defined. This role might be played outside the classroom under the term collaborative work, which may involve contacting people such as the publisher, editor or other translators. In fact, these people could be characterized as experts in their respective fields. Also, since translators are not assumed to have knowledge of specific editorial conventions, regular meetings with the editor might be necessary. However, although contacting these experts might be a good idea, it should be noted that they are not always easily accessible.

In TS, the term ‘expert’ is mostly associated with a working professional in the field of the text to be translated. It is suggested that occasionally the role of the expert could be played by the translation teachers, at least as regards the fields in which they have specialized knowledge, often acquired as a result of their working experience. A risk associated with the expert and the translator is the likelihood of one trespassing on the other’s field. Nevertheless, this risk could be alleviated if all stakeholders work collaboratively. Whenever subject experts are invited into the classroom, they should be prepared for their specific duties. This preparation may include the type of expert knowledge offered and the exact moments of intervention or interaction with the teacher and the students. Yet, in all these cases, the students should be properly informed in advance and discuss any concerns they may have about the intervention of a third party. Teachers might also assume the role of the expert (because they happen to have expertise in the subject of the text to be translated) but it may not always work as planned (Damaskinidis 2015).

Another line of research could usefully look at the way the different reading paths affect the translation process. The translation could be monitored by using eye-tracking software (Dam-Jensen and Heine 2009) as a method for examining students’ eye movements, so as to disclose their visual attention, in other words, trace their reading paths. By studying these reading paths, knowledge may be obtained about where students’ focus their attention in the translation process. The order in which the various verbal and non-verbal elements are observed, and the corresponding time spent there, could be used by the researcher/teacher as anchoring points for feedback and further discussion. As a follow up activity, the same multimodal text could be split up into its various components and given in a different order to other students. It would then be possible to match the translations of the modified texts against the original data-texts by using the eye-tracking software. An eye-tracking-based research could also give answers to the question of how long translators (should) delay the translation process in order to consider non-verbal elements.

The complexity of multimodal text’s layout lends itself to the employment of photo elicitation techniques (Harper 2002). The students are divided into groups, where each is given the same data-text for translation, but with a different photograph accompanying it. Care should be taken to ensure that all photographs are related to each other and each one to (some of) the verbal elements. This could be followed by a focus group discussion where all students explain their translation choices. The students should feel confident about their choice of verbal/non-verbal relations and feel able to challenge the teacher’s and the other students’ choices, and at the same time, be ready to defend their own. This may lead to the production of TTs that vary significantly. Even if these TTs are not considered as genuine translation work in professional terms (for example, the translator would be paid for that job), their comparison could become a springboard for a fruitful and constructive dialogue between the students and the teacher.

A follow up activity is related to the importance of localization in TS. For example, in a translation activity Greek students were asked to replace the photograph of an English advertisement so that the Greek version is localized for their own culture (Damaskinidis 2015). Briefly speaking, the depicted baseball player was replaced by athletes who would be more familiar to a Greek audience (for example, football goalkeeper, taekwondo, boxer). This activity engaged them in lengthy discussions about potential relations between verbal and non-verbal semiotic elements and helped them to reach their final translation solutions. Thus, the translation does not end with the verbal part but with delivering the final product, according to the new requirements for the new intended audience. Such a localization scenario calls for combining language translation with photo-journalism, further entailing new procedures such as functional and linguistic testing of the localized advertisement. In a westernized context, it is common practice for advertisements to become globalized by keeping intact the photograph and localizing the text. This is based on the assumption that the advertisement images will become a kind of fashion icon. The challenging requirement of changing the photograph rather than the text goes beyond the normal call of a translator and as a result a new specialized sector emerges in close association with the advertising industry, which reflects the complexity involved in making an advertisement global-ready.

The extent to which the suggestions for curriculum development are realistic and how they will be fitted in depend on various factors, such as the mode of teaching, the type of class and the availability of resources. While some suggestions may be realistic throughout the teaching period, some may have to be fitted into practical training periods. For example, it could become a standard procedure to bring together students and experts in various disciplines, such as law, technology, physics or architecture. This could take the form of a formal invitation to an architect, for instance, to join the class and discuss the translation of an architectural sketch. Alternatively, it could become a task for students to contact an advertising company outside the class to discuss the localization of advertisements. On the other hand, a specific training period could be arranged where students are placed in a translation agency, or a company, so as to prepare a translation portfolio on a specific translation area or subject.

6. Conclusion

Translation is associated with visual literacy mainly through multimodality, a concept which has been recently conceived, and as such has only now begun to find its place in TS. It has been shown that multimodality has begun to be recognized in TS, through concepts such as ‘multimodal,’ ‘intersemiotic’ and multidimensional.’ However, this stems from the proliferation of multimodal texts that have found their way into translation, rather than from TS seeking to expand as a field.

A problem is that logocentric theories have dominated TS thus far. The fact that the verbal element was, and in many cases still is, the main mode of communication of concern to TS makes multimodal approaches to translation a secondary issue. Although intersemiotic approaches to translation took into account the non-verbal mode, a major breakthrough has been the incorporation of multimodal and multidimensional approaches to verbal-visual interactions which have moved translation training even further forward.

It has become clear that translation in general, and from a multimodal perspective in particular, cannot be limited to translating words in one language into words in another language. Reading and analyzing multimodal texts require an array of non-verbal skills, such as the reading of non-verbal elements, which could be facilitated by employing a kind of visual grammar. However, since this type of grammar is still under development, it is questionable whether this facilitation is really reliable. Also, a basic knowledge of other fields, such as graphic design or photography, goes beyond the translator’s usual skills. In fact, the skills that may be needed are as many as the specialized domains of the multimodal text to be translated. In other words, visual literacy is definitely an asset, if not a requirement, for the translation of multimodal texts.

In terms of the creativity required to make associations between verbal and non-verbal semiotic modes, translation teaching does not focus to a great extent on the education of students as creative, intelligent and competent human beings. This lack of focus could be ascribed to the inability to distinguish between translator training and translatoreducation for university translation programmes. While the former is devoted to the specific skills required by the translator, the latter balances between translation specialist competence and language teaching and learning.

The cultural conditions of the media in the European Union context are such that it would be necessary to develop professionalised training for translators in order to promote European cultural unity through cultural diversity and to maintain the cultural richness in today’s multilingual societies. While in multimedia environments there are many cultural differences, in the case of newspapers and magazines this sort of publishing overlaps in the EU context. For this purpose, interdisciplinary and international approaches are to be favoured. Course contents and principles of practice could be developed and homogenized to allow for a smooth exchange of translator trainees between educational institutions. On other occasions, the content and principles could be context-specific, or even context-differentiated, in view of the on-going European enlargement.

Parties annexes