Translator’s Frequent Lexis Store and Dictionary Use as Factors in SLT Comprehension and Translation Speed – A Comparative Study of Professional, Paraprofessional and Novice Translators

Ronowicz, Eddie; Hehir, Joanna; Kaimi, Toshihiro; Kojima, Keiko; Lee, Deok-Shin

doi:https://doi.org/10.7202/011002ar

Introduction

There were three objectives of the project presented in this paper. First, Think Aloud Protocols (TAP) as a research method were to be tested; secondly, we regularly involve our research methods students in “hands-on” research work and this was an excellent opportunity for the supervisor and the students to work as a team; thirdly, there was a heuristic aim: we asked some research questions and attempted to obtain answers to them by analysing TAP data.

The paper presents the results of two separate pilot experiments carried out in the first and second semesters in 2002 with two groups of students enrolled in a course on research methods in translation studies and will be followed by a larger project.[1] The final part also includes some methodological observations about TAPs and other methods of process-oriented research based on our own observations, as well as those of researchers who have done this kind of research before us.

Comprehension of Source Language Text

Translator knowledge and skills involved in SLT comprehension

We assume that, unless a source language text (SLT) has been well understood, a good translation cannot be produced. Strategies employed by individual translators in trying to process a source language text will depend to a large extent on their knowledge of the source language and culture, as well as their knowledge of the world in areas covered by a specific text and its elements. Indeed, as Gile points out, “In order to account more fully for comprehension, the basic comprehension requires another element besides knowledge of language and extralinguistic knowledge, namely deliberate analysis” (Gile 1995: 80). Gile (1995: 78-80) illustrates the relationship as follows:

The “knowledge of the language” element of the formula above depends on internal resources the translator has available in memory and, should memory search fail, on skill in using external resources. Using Roger Bell’s (1991) terminology, our interest in this study is in Frequent Lexis Store (FLS) and in the use of the dictionary as a means of gaining the best possible comprehension of the Source Language Text (SLT) and finding the closest possible equivalents to be used in the Target Language Text (TLT).

Frequent Lexis Store

According to Bell’s model of translating (1991: 59), the initial stages of processing the source language text include visual word recognition, followed by processing in what he calls the “syntactic analyser.” The syntactic analyser contains a Frequent Structures Store (FSS) and Parser, as well as a Frequent Lexis Store (FLS) and a Lexical Search Mechanism used in this order as seen in figure 1 depicting the relevant fragment of the model. Bell defines the Frequent Lexis Store as a “(…) mental (psycholinguistic) correlate to the physical glossary or terminology database, i.e., an instant look-up facility for lexical items both words and idioms.” (Bell 1991: 47)

According to Gile, “Sentences in informative Texts can be represented as network-like structures consisting of three types of components:

Nouns and noun-phrases that indicate persons, objects, ideas, actions, etc. These are referred to here with the term Nominal Entities (NE’s)
Adjectives, adjective-like words, and clauses that describe these persons, objects, etc. (“big,” “small,” “expensive,” “resistant”), as well as statements of existence, disappearance, growth, etc., about them (“X exists,” “Y has grown,” “Z proliferates”). This type of component is referred to with the term Attribute (A)
Structures and occurrences of rules of grammar (declensions, word order, etc.) which establish logical or functional links between these persons, objects, or concepts (A is compared to B, A acts on B, etc.), This third type of component is referred to here as a Link (L)” (Gile 1995: 89)

Gile (1995) rightly points out that, depending on the individual’s changing experiences with a variety of domains tackled in translation assignments over longer periods of time, the contents of FSS and FLS, which in his analysis are conflated into one store, are variable over time. Gile explains this variability using The Gravitational Model of Linguistic Availability which “consists of a variable part and an invariable part. The latter refers to language elements the availability of which is assumed to be constant or to vary very slowly. This applies to the most basic rules of grammar (basic conjugations, the formation of plurals, etc.) and to a small number of the most frequently used words in the language. The variable part is larger by several orders of magnitude, as it includes at least dozens of rules and many thousands of words and idioms” (Gile 1995: 217).

If we adopt Gile’s classification, we would expect the Frequent Lexis Store of a translator to contain a certain number of Nominal Entities and Attributes, depending on this translator’s experience and knowledge of the source language and domains s/he covers on a regular basis at a given point in time and the Frequent Structure Store to contain Links. One would also expect that more experienced translators will have a larger and more diversified FSS and FLS, which should influence the speed and quality of their performance.

Dictionary use

If neither the FLS nor the Lexical Search Mechanism provide an answer to a translation problem, the translator has to resort to external resources – dictionaries being the first port of call in case of lexical difficulties. Empirical Think-aloud protocol research using the use of dictionaries by professional and novice translators over the last 15 years or so has produced evidence which does not lend itself to generalizations easily.[2] Some of the evidence which is usually referred to as evidence on “novice translators” has been collected from advanced language learners (e.g., Krings 1986, Gerloff 1988, House 2000), some from undergraduate or postgraduate first year translation students (e.g., Jääskeläinen 1990, Kiraly 1995). Also, depending on which study one deals with, the category of “professional translators” includes subjects of anything from one or two years experience (e.g., Kiraly 1995) to translators with well over 10 years professional experience (e.g., Jääskeläinen 1990, Jensen 1999). This raises the question of whether subjects who participated in different studies are comparable for the purpose of generalizations. Nevertheless, past research has generated interesting conclusions and we will now look at their findings.[3]

As far as we are aware, the first empirical study that looked at use of reference books was that of Krings (1986), whose subjects were advanced language learners. Krings found that “The main subtypes of comprehension strategies are inferencing and use of reference books. Most of the subjects immediately made use of dictionaries when they encountered lexical items they didn’t know. (…) Inferencing appeared whenever for some reasons the use of reference books was impossible or turned out not to be helpful” (p. 270). Krings also commented on some decision-making strategies he observed which seem to illustrate well attitudes we may find in non-professionals: “most of the subjects tended to resort to specific types of decision-making strategies that might be labelled ‘translation principles.’ (…) They are reducible to imperatives such as, to give a few examples: ‘If all competing potential equivalents turn out to be equally appropriate or inappropriate, take the most literal one!’ or alternatively: ‘Take the shortest one!’ A further principle is concerned with reference books stating: ‘If all equivalents concerned are in the dictionary, take the one that precedes the others!’” (p. 273)

Jääskeläinen (1996) used the results from two unpublished dissertations (Gerloff 1988 and Jääskeläinen 1990) to demonstrate that professional and semi-professional translators who produce good quality translations consult dictionaries more often and in more sophisticated ways than novice translators, who “(…) problematise relatively little. As a result, they translate quickly and effortlessly (and perhaps wrongly, depending on the difficulty of the task), i.e. novices are blissfully unaware of their ignorance” (p. 67).

Dancette’s (1997) findings seem to confirm this. Her subjects were postgraduate translation students and she found that the subject who produced the lowest quality translation “(…) opens dictionaries whenever she does not understand a term” (p. 101). The two subjects who produced better quality translations, on the other hand, “open dictionaries once a hypothesis on meaning has been formulated, to monitor a translation choice or an interpretation, or to find alternate equivalents or synonyms” (p. 101).

Jensen (1999), who compared novices, young professionals and experts (over 8 years experience) found that “The use of dictionaries decreased with increased experience, and when comparing young professionals with the expert group we find that the experts had only half as many dictionary look-ups as the young professionals. The only dictionaries used were bilingual dictionaries, except by one of the non-translators” (p. 113). Hannelore Lee-Jahnke (1997) whose study looked, among others, at numbers of dictionary consultations reports a similar finding (cf. pp. 164-165).

Livbjerg and Mees (1999 and 2002), whose subjects were five of their most competent postgraduate translation students, asked them to translate a text first without access to reference books, then they gave them access to reference books and allowed them to work on the same translation (1999 study). A repeat experiment in which 5 comparable translation students were asked to translate the same text, but were given dictionaries from the outset was carried out later (2002 study). The results for Group 1 (1999 study), which were almost identical to those observed in Group 2 (2002) confirm some of the earlier findings about translation students. Of the total of 121 translation units[4] analysed in Group 1, 70 were handled without dictionaries (58%), 51 were looked up (42%), in other words, students did consult dictionaries quite a lot (Livbjerg and Mees 2002: 158). However, further analysis demonstrated that there were “only 12 units (out of a total of 121 that were processed) in which the consultation of dictionaries resulted in an error being changed into a correct solution” (Livbjerg and Mees 2002: 162). It is no wonder, then, that the authors conclude that the value of dictionaries to students was rather limited, even if they do fare better with them than without (p. 169), a finding which confirms Krings’ (1986), Jääskeläinen’s (1996), Dancette’s (1997) conclusions.

Furthermore, if we consider that professional translators usually do produce better translations than students, one has to conclude that it is not the frequency of using dictionaries, but strategies of using them that are crucial in both comprehending the SLT and producing a good quality TLT. Similarly to Jensen’s (1999) study, our study compared translation students, young professionals (or paraprofessionals) and professionals.

Purpose of study

The literature review suggests that there are differences between the frequency and manner of dictionary use between novice, paraprofessional and professional translators. In order to confirm previous results and account for these differences, we proposed the following specific aims for our study:

To assess Think-aloud protocols as an introspective research method into translation processes.
To assess the translator’s Frequent Lexis Store prior to performing a translation task and to find what, if any, influence this has on work with dictionaries and the speed and quality of translation.
To examine the frequency of using dictionaries and see if this, as well as the content of the FLS influence the speed and quality of translation.
To examine strategies of using dictionaries and their influence on the quality of translation
to look for evidence of source language text analysis during the translation process and possible links to the way dictionaries are used and to speed and quality of translation;
To compare the performance of novice, paraprofessional and very experienced professional translators.

Think-aloud protocols are a qualitative method involving case-studies, and hence we decided that a methodologically sound attitude would be to formulate aims only and try to formulate some hypotheses after the results were analysed rather than before it.

Experimental design

The subjects

Following Kiraly’s comments on difficulties in pointing out significant differences between his “novice” and “professional” translators’ performance (Kiraly 1995: 89 ff), we tried to secure subjects with significant differences in length and type of experience in translation and divided them into three groups using the Australian National Accreditation Authority for Translators and Interpreters terminology as follows:

Novice translators: our first semester students of English / Chinese, Japanese, Korean and Spanish translation – 5 subjects in all.
Paraprofessional translators. Our completing students with good or very good final results and former students already working in the profession but with less than 3 years experience: our third semester and former English / Chinese, Japanese, Korean and Spanish translation students – 4 subjects in all.
Professional translators: our staff members and one outsider with more than 3 years experience (actually between 4 and 18 years of experience) in professional translation: 5 subjects (two English / Japanese translators and one each English / Chinese, Korean and Spanish translators (5 translators in all).

A pre-test questionnaire asking about personal, educational and professional background (see Appendix 1) was administered to all subjects to obtain information allowing us to allocate them to the three groups mentioned above. For reasons beyond our control (see 5.1. below for explanation) we could only use TAPs done on Japanese and Korean subjects. Their personal information is summarized in Table 1 below.

The text used in the translation task

The text to be translated by the subjects was selected from The Economist, April 6, 2002 and the section chosen for translation was 378 words in length (see Appendix 2). It was chosen because it uses a good deal of specialist economic terminology, the length is appropriate to the time limit for the translation task (90 minutes) and it seemed a difficult enough article to be quite challenging to the translator. We assumed that, as a result, we would have examples of subjects resorting to a variety of translation strategies other than just accessing a frequent lexis and frequent structures stores and that they would have to demonstrate external knowledge relevant to the domain of the text in order to produce a high quality target text.

Content of the subjects’ Flexible Lexis Store prior to the translation task

The content of the subjects’ FLS within the domain covered by the text was tested by the administration of a vocabulary test of 40 words to each subject prior to the translation task (see Appendix 3). The words to be included in the test were selected by MA student – investigators whose level of knowledge may be described as that of paraprofessional translators, i.e. equivalent to the second category of our subjects. The text, which was not known to any of them except for the student investigator responsible for finding it, was shown to them for the first time in class and they were asked to read it and select all those words and phrases which they thought could cause them difficulty in translation. A common list of some 20 difficult items was then created and another 20 easier words from the text were added, then all 40 words were randomized. The test was then stored electronically in a program that allowed flashing each word for 7 seconds on a computer screen one after another without interruption (for the discussion of this type of methodology, see De Groot 1997: 34-38). The limited time of 7 seconds was allowed to make sure that subjects relied on their FLS only and did not have enough time to activate the Lexical Search Mechanism. The subjects were asked to read the word on the screen aloud, then write down its meaning in their first language.

The translation task

The experiment was carried out in the on-campus audiology studio with a computer monitor for the vocabulary test, a video player and monitor for demonstration purposes, a camera and a microphone to record the subject’s behaviour and voice, a desk and chair for the subject, pen and paper, the text to be translated and a copy of the whole, unabridged article as reference, as well as, depending on language involved and availability, between 7 and 9 different types of dictionaries. The subjects were asked to translate part of the English text taken from The Economist into their native tongue (Chinese, Japanese, Korean, Spanish) verbalizing everything going on in their minds while they are translating. 90 minutes were allotted for the translating task.

The results and discussion

Limitations on data available for analysis

14 TAPs were recorded altogether in both experiments but, for a variety of reasons, only 9 TAPs (six Japanese from experiments 1 and 2 and three Korean from experiment 2), properly administered and analysed and accompanied by the vocabulary pre-test could be used for this presentation[5].

Some of the data obtained could be expressed quantitatively, other data derived from observation of the subjects’ behaviour could only be expressed qualitatively in terms of descriptive statements. The presentation of findings is restricted to those which we considered to be backed well by properly collected and analysed evidence.

Vocabulary test results (FLS content)

All subjects’ FLS was measured by the vocabulary test and demonstrated clearly that, as expected, two of the professionals with extensive experience (more than 15 years) had all or almost all the words in their FLS ready for use, even if some of the words required further checking in the context of the specific translation task. The third, less experienced professional (P2Jap – 4 years) had virtually the same level of content of the FLS as the three paraprofessionals. The results have been summarized in Table 2 below.

Use of dictionaries during the translation task

We obtained reliable data on the use of dictionaries by six subjects, three Japanese and three Korean. Each subject had access to 9 (Japanese) or 8 (Korean) different types of dictionaries (smaller and larger bilingual dictionaries, monolingual dictionaries in both languages involved and in English, and specialized dictionaries of economic and finance terminology).

Table 3

**Dictionary consultations (6 subjects)**

If we look at the raw numbers in Table 3, it is clear that novices have used dictionaries much more frequently than paraprofessionals and professionals. There does not seem to be any significant difference between the number of consultations made by paraprofessionals and professionals, except for one professional (P3Kr – only 17 consultations), who is also a very highly ranked international conference interpreter and may, for this reason, operate differently than non-interpreter professional translators – this would need to be checked.

Moreover, as Table 4 and accompanying graph demonstrate, there is a clear correlation between the content of FLS and number of dictionary consultations: the higher the content of FLS, the fewer dictionary consultations were needed (professionals).

Table 4

**FLS and dictionary consultations**

These results appear to be in line with previous findings in most of the studies mentioned above that novice translators consult dictionaries more frequently than professionals. It also suggests that the higher number of dictionary consultations by novices is directly related to the lower content of their FLS’s. We then attempted to check if patterns of dictionary use were different in the three categories of translators. Table 5 presents a summary of types of dictionaries consulted by the same six subjects.

Table 5

**Type of dictionary consultations (6 subjects)**

In the first place, Jensen’s (1999) finding that bilingual dictionaries are most popular has been confirmed in slightly different conditions: in Jensen’s study the subjects used dictionaries of their choice, in ours great care was taken to provide subjects with a representative sample of standard reference works to choose from (8-9 different types of dictionaries). In our study, a bilingual dictionary was used in 56.6% of all dictionary consultations.

The results of patterns of dictionary use by the three categories of subjects are inconclusive. We can, however indicate some trends, which would have to be checked carefully in further studies.

None of the novices, who obviously had comprehension problems with the English source text consulted a monolingual English dictionary and they seldom checked more than one dictionary. Even if they did, when they still did not comprehend the SLT or find good equivalents, they would pick one, which often was inappropriate in the context and use it without hesitation. This seems to be a probable explanation for Livbjerg and Mees (1999 and 2002) finding that dictionaries were of limited use to their student subjects.

Professionals and paraprofessionals, on the other hand, made use of monolingual and specialist dictionaries as they faced either comprehension or production problems. Out of the total of 23 consultations in monolingual dictionaries. 22 or 95.7% were made by professional and paraprofessional translators and out of 21 consultations in specialist dictionaries, 15 or 71.4% were made by the same two groups. In fact, the two very experienced professionals in particular checked on words they had in their Frequent Lexis Store, which indicates they sensed potential problems and attempted to make sure that their comprehension of the source text is full or, in case of production of the target text, the equivalent they found was the best available at the time. Both professionals and para-professionals were very cautious about the accuracy and naturalness of word usage, they did not always trust equivalence obtained from the dictionaries and occasionally commented during the sessions that even though they found the meaning of a word in a dictionary, they had to modify the wording into an appropriate or natural one to make the target text more readable.

The above analysis is confirmed by the original target texts produced by subjects during TAPs. The target texts of novices had very few crossings out in contrast to professionals and paraprofessionals, who have crossed out some equivalents, replaced them with others, sometimes two or three times. In other words, professionals and paraprofessionals tended to work and re-work their target text systematically until they were satisfied with the translation.

Evidence for text analysis

The TAPs also provided us with some evidence indicating that professional and some paraprofessional translators have spent time analyzing the text to be translated. All of the four professionals (including the Chinese translator, who is not included in the other data) and two of the paraprofessionals (P1Jap, P2Jap, P3Kr, P4Ch PP1Jap, PP2Jap) took time to read the whole text before they started working on it unit by unit. Their comments while reading suggested they were trying to identify the type of text they were dealing with and, in case of two professionals (P2Jap and P3Kr) and one paraprofessional (PP1 – a lady of 20 years experience but limited amount of professional work done in the past) used the reading to identify problems, make notes on the margins and between lines, indicating very clearly that they were analysing the text not just as a whole, but smaller problematic translation units as well[6]. The other two professionals (P1Ch and P1Jap) tended to use a similar method of analysing the text, but paragraph by paragraph, as they were working and clearly moved backwards and forwards from the fragment they were working on to clarify the meaning in context. None of the students engaged in this sort of activity. The data seem to suggest, that the more experienced translators are, the better they are aware of the importance of text analysis. Unfortunately, we have not investigated details of this process in sufficient depth to make any further comment.

FLS and the quality and speed of translation

We have not obtained enough reliable data in the study to make an informed comment about the relation between the FLS or strategies of dictionary use and the quality of translation. We have, however been able to demonstrate a correlation between the content of the FLS and the speed of translation for all the 9 subjects whose data we could use. Namely, as the table and graph below illustrate, it would seem that, the higher the content of the Frequent Lexis Store in the domain of the source language text, the higher the speed of translation.

This does not mean to say that a large FLS is the only factor influencing the speed of translation. The fact that in case of three subjects the correlation is not as straightforward as with the remaining six suggests that there may be other factors at work. There is no doubt, however, that fast access to at least one meaning of almost every word used in the source language text helps by leaving more time for other activities, such as text analysis and, in case there is any doubt about the meaning of a particular item, dictionary consultations appear to be more targeted in subjects with a larger FLS.

Table 6

**FLS content and speed of translation**

Concluding remarks

Using students as subjects of TAPs and other empirical process-oriented studies.

It seems that the results we obtained, as well as those obtained in previous studies, lend very strong support to Jääskeläinen’s (1996) suggestion that students, especially, we would add, non-translation language students and 1st semester translation students, seem to be blissfully unaware of their ignorance. They usually do no know how to use larger and/or specialized dictionaries in a targeted way as a result of their lack of ability to analyse the nature of the problem and select an appropriate solution. In other words, we have confirmed the obvious. Consequently, we would have to question the value of further studies investigating so called ‘novice’s’ use of translation aids.

If we are to teach our students how to use translation aids effectively, we should probably accept that, at least in the area of strategies of using dictionaries further studies of non-translators cannot provide us with much more useful information, and that we should now focus on studies of professional translators using various methodologies. In order to gain the necessary knowledge to attain this goal, we probably need to re-examine anecdotal (written from experience), theoretical and empirical studies of strategies of use of dictionaries and other reference sources by professionals and design empirical studies to obtain supporting evidence (cf. also Fraser 2000).

All the results of studies of paraprofessionals (otherwise referred to as young professionals or semi-professionals) and professionals quoted above suggest that in the case of paraprofessional and professional translators use of translation aids is characterized by some particular routines or strategies. The problem has been discussed theoretically in many popular translation textbooks for students and teachers (cf. Gile 1995), but very few professional translators have had their strategies of using translation aids studied empirically. The logical conclusion would be that there is a need for continued studies of paraprofessionals, who these days are usually the products of our training – to find out what are the effects of this training; and professionals of considerable experience or experts – to find out what exactly it is they do to achieve efficiency and quality that characterizes their translation work.

We realize from our own experience that it is much harder to secure larger numbers of subjects from among experienced translators for a variety of reasons. However, since the mid-eighties most researchers interested in process-oriented studies have stated one way or another their interest in the translator’s “black box” and what is happening there (cf. e.g., Borsch 1986, House 1988). We strongly believe that, by definition, first semester students and non-translator language students are not translators – even if they would like to become ones in the future, and that studies of paraprofessionals and professionals would yield more data relevant to the “box,” which still seems to be black rather, than some shade of grey.

Think-aloud protocols as a research method

It was encouraging to find that our concerns about TAPs as a research technique were shared by some experienced researchers (Toury 1991, Jääskeläinen 1996: 61, Jääskeläinen 2000, Bernardini 2001). While empirical knowledge is, undoubtedly, the most reliable type of knowledge, one must not forget that one of the reasons for its reliability is methodological rigour, including clear statements of objectives, careful development of testing and analytical instruments, as well as a good understanding of the advantages and limitations of different experimental designs. (cf. Toury 1991, Jääskeläinen 2000 and Bernardini 2001).

Introspective methods are, by the very nature of the experimental design, qualitative case studies. It follows that they are not hypothesis testing studies, but hypotheses generating ones and further quantitative research is needed to test hypotheses generated. One way of improving the situation would be to share TAPs from different institutions, a bit like first language acquisition scholars shared evidence from many different countries and institutions, as suggested by Jääskeläinen 2000 and Bernardini 2001. This, however, would require the development of shared methodological principles between a number of institutions, otherwise the data will not be comparable and storable in the form of a computer corpus.

Another interesting development is use of devices providing quantitative data alongside think-aloud protocols, such as Translog, a software recording keyboard activity and timing that activity (cf. Jakobsen 1999). Three of the studies quoted above have used Translog and think-aloud protocols (Jensen 1999, Livbjerg & Mees 1999 and 2002) providing the authors with two sets of simultaneously recorded data and resulting in a more rigorous interpretation of the results.

Furthermore, improvement and some degree of standardization of ways of assessing the quality of translations produced in TAP studies is needed. One gets the impression that we have somehow forgotten that there is a considerable body of knowledge produced by researchers who focused on texts rather than the mind and on quality assessment (see e.g., House 1997). Although there are some notable exceptions (e.g. Kiraly 1995, Livbjerg & Mees 1999 and 2002) most studies, including our own, that looked into the question of the quality of the target texts produced by their subjects have used rather impressionistic ways of assessing and describing it.

Finally, it seems that the popularity of TAPs in the last decade or so, has somewhat overshadowed other experimental methods and techniques. TAPs are not – and should not in our view – be the only method of investigating cognitive processes in translation. We believe that some of the existing hypotheses derived from theoretical models or introspective studies may lend themselves to empirical verification in questionnaire-based studies and, in particular, in classical single-hypothesis testing experiments involving strict control of variables (cf. Ronowicz, forthcoming) in order to achieve the ultimate goal of having a fully, or at least partially, empirically verified theory of translation some time in the future.

C	stands for comprehension
KL	stands for “knowledge of the language”
ELK	stands for “extralinguistic knowledge”
A	stands for “analysis”

Translator’s Frequent Lexis Store and Dictionary Use as Factors in SLT Comprehension and Translation Speed – A Comparative Study of Professional, Paraprofessional and Novice Translators

Abstract

Résumé

Introduction

Comprehension of Source Language Text