Quality Control in the Subtitling Industry: An Exploratory Survey Study

Robert, Isabelle; Remael, Aline

doi:https://doi.org/10.7202/1039220ar

1. Quality management, quality assurance and quality control

Quality is a central issue in translation research, practice and training today. However, the topic is studied from different angles, and despite, or possibly because of, the great interest from different players, the terminology that is used to refer to central aspects of translation quality is not always homogeneous.

This article will focus on a number of issues related to subtitling quality; more concretely, it will report on a survey conducted among professional subtitlers in 2013. Going into a detailed discussion of terminological issues therefore lies outside its scope. However, we will clarify the terms we use and explain our interpretation of them below.

Three key terms that recur in the literature are quality management, quality assurance and quality control, and although they are sometimes used as synonyms, for us they are not. We will consider quality control to be an aspect of quality assurance, and quality assurance to be a part of quality management, establishing a hierarchy between the three concepts. We share this approach with Tricker (2014: 3-4), who defines “quality” as consisting of two separate yet connected activities, quality control (QC) and quality assurance (QA), which are both part of quality management.

Quality control is the part of quality management focused on fulfilling quality requirements, whereas quality assurance is that part of quality management that focuses on providing confidence that quality requirements have been fulfilled. Quality assurance is implemented in the three stages of the translation process as described by Drugan (2013: 77-80) and illustrated in Figure 1. The figure also shows that QC is carried out in the pre-translation stage when the project manager, for example, prepares the source files and quality controls them, but also in the translation stage by the translator, and in the post-translation stage by the reviser before project completion, that is, before delivery to the client.

Figure 1

**Quality Assurance in the translation process**

The definitions to which we subscribe are also in line with Mossop’s definition of quality control (2014: 128) as a contribution to quality assurance, to be carried out before delivery to the client, whereby quality assurance is understood as

the full set of procedures not just after (as with quality assessment) but also before and during the translation production process, by all members of a translating organization, to ensure that quality objectives important to clients are being met
Mossop 2014: 129

The author does not make the distinction between quality assurance and quality management but seems to consider the two terms as equivalents.

A central document for our research is the European standard for Translation Services EN 15 038 (European Committee for Standardization 2006)[1], which is a quality standard from order to delivery. Its purpose is “to establish and define the requirements for the provision of quality services by translation service providers” (European Committee for Standardization 2006: 2). The standard encompasses not only the core translation process but also all other related aspects involved in providing the service, as defined above, including quality assurance and traceability. Most crucially for our research, the standard states that each translation has to be fully revised by someone who is not the translator, after having been “checked” by the translator. Therefore, it imposes a very high degree of “quality control” as defined by Mossop (2014: 116). Even though the standard is not entirely clear on the way in which the revision should be carried out, that is, by means of a full comparison of source and target text or through a more limited target text reading, it seems to imply that a full comparison is required (Robert and Van Waes 2014).

To return to our study, we aim to look into the degree to which quality is monitored in audiovisual translation and, in particular, subtitling. Do translation agencies offer subtitling or do professional subtitlers work according to the standard? Do they work in accordance with the Code of Good Subtitling Practice (Carroll and Ivarsson 1998), stating that “each production should be edited by a reviser/editor”? More concretely, we will provide a provisional map of the current quality control and assurance practices in subtitling in Europe and beyond. We will concentrate on:

quality control during the translation process proper, that is, the revision of the subtitles by the subtitler (“checking,” according to the standard);
quality control after the translation process proper, that is, the revision of the subtitles by another subtitler (“revision”);
aspects of quality management which can contribute to quality, even before the translation process proper has started, that is, human resources (profile of the subtitlers) and technical resources (material available to carry out the subtitling task).

In other words, we will concentrate on quality control, and on aspects of quality assurance.

First, Section 2 will focus on revision as a form of quality control. Section 3 will then detail our participant-oriented research method and our use of an internet-mediated questionnaire, which we have chosen for the core of the study. Here we investigate how subtitlers and their clients carry out quality control and quality assurance in their daily subtitling practice. Section 4 summarises our findings and compares them with those of another recent study by Kuo (2014a) on the same topic. Section 5 contains the conclusions and suggestions for further research.

2. Revision as part of quality control

2.1. Revision: a definition

Just like the concepts relating to translation quality discussed above, the concept of “revision” is not always understood in the same manner. Revision may be broadly defined as the process of looking over a translation to decide whether it is of satisfactory quality and making any changes that are required. However, this apparently simple procedure covers a range of activities, which will be considered here on the basis of who performs them: the original translator, a second translator, or a non-translator. There is no consensus in the English-speaking world about how to call these activities carried out by different agents. They may be referred to as “revise, re-read, cross-read, proofread, review and quality-control,” to name the most frequently occurring variants (Mossop 2014: 116).

This terminological confusion about the concept of revision has been underlined by several scholars, such as Künzli (2005), Lee (2006), Martin (2007), Scholdager, Wølch Rasmussen and Thomsen (2008), Hernandez-Morin (2009a, 2009b), Mossop (2011), Robert (2008, 2012, 2013), Robert and Van Waes (2014), Drugan (2013), and very recently again by Künzli, who proposes a new definition:

Review of an initial version of a human-translated text – or parts of such a translation – carried out by another translator and consisting of a more or less comprehensive comparison between the source text and the target text and of the execution of all the corrections considered necessary before the translation is delivered to the client.
2014: 3 (ranslated by the author)[2]

It is a definition that we subscribe to in all but one respect, namely, the fact that in Künzli’s view revision can also be carried out on part of the translation rather than on the entire translation.

In this paper, the term “revision” will be used to refer to an activity that covers the entire text only. An operation that consists of revising just a part of a target text, sometimes called “spot-check” (Mossop 2014: 159), is another type of quality control (see also Brunette 2000), whereas the revision of one’s own translation will be referred to here as “self-revision,” which the EN 15038 calls “checking” and which obviously happens before revision proper. In terms of translation quality management, revision is an example of quality control, since it is conducted on the product in a non-final version, that is, the revision takes place before the translation is delivered to the client (Brunette 2000).

2.2. Quality parameters in translation revision[3]

Revision parameters are similar to the quality parameters applied to the evaluation of a translation, which means that, inevitably, they are relative and subjective because, as Graham writes,

[…] there is no such thing as one single correct translation of any text as opposed to one-for-one correct renderings of individual terms; there is no single end product to serve as a standard for objective assessment. Indeed, there is more than a grain of truth in the old adage that ten equally good translators, translating the same text, will produce at least 12 equally good translations, all of which are entirely different in all but content. […] Consequently, no absolutely impartial standards or sets of criteria can be applied to measure or assess the quality of a translation.
1989: 59

Quality parameters are also relative because they can depend on several factors, such as time, place, context of communication, dominant theories of translation, genre, type and purpose of the text, target reader, etc. (Horguelin and Brunette 1998: 12) Still, there is some consensus among scholars about basic criteria, as noted by Horguelin and Brunette: “Cette relativité admise, il est possible d’arriver à une définition de la traduction qui constitue une synthèse des critères contemporains généralement acceptés – et peut donc servir de guide pour la révision.” (1998: 12)

However, in spite of such a broad consensus, the perception of quality in translation still depends to no small extent on the viewpoint of each participant in an assessment. As Gile (1995: 34) points out, even if there is a consensus on some quality criteria that are more or less independent of the context (that is, ideational clarity, linguistic acceptability, terminological accuracy and fidelity as well as professional behaviour), the weight given to them by individual raters may vary.

As for revision, Robert (2012: 27-39) shows that there are several typologies of revision parameters in the literature, the most complete of which is the one proposed by Mossop (2001, 2007, 2014). It is a synthesis of the author’s experience in revision and of earlier proposals by authors such as Graham (1989: 66-67), Sager (1994: 239) and Horguelin and Brunette (1998: 11). The typology takes both self-revision and revision on board. It consists of four groups of parameters related to (1) Transfer, (2) Content, (3) Language and (4) Presentation. Each group is subdivided into subgroups as follows: Transfer covers “accuracy” and “completeness,” Content covers “logic” and “facts,” Language includes “smoothness,” “tailoring,” “sub-language,” “idiom” and “mechanics,” and, finally, Presentation covers “lay-out,” “typography” and “organisation.”

Surveying the literature, Robert (2012: 27-39) found out that later typologies generally include fewer but broader parameters, to make the analysis more feasible. Brunette, Gagnon and Hine (2005) distinguish between four parameters; Lee also has four (2006: 418); Hansen suggests five (2009: 278); the EN 15038 has seven (European Committee for Standardization, 2006: 11), and Parra Galiano lists nine (2007). All of these typologies are proposed in handbooks or review articles, but two of them are based on a survey of professional revisers.

The first was carried out by Schjoldager, Wølch Rasmussen and Thomsen (2008), who conducted an exploratory survey of the translation industry internationally and in Denmark. Their aim was to gain insight into professional précis-writing as well as revision and editing. Their study includes a survey of relevant norms and concepts, with a view to developing a pilot module on précis-writing, revision and editing for the European Master in Translation (EMT). In their data analysis, the authors identify a set of objectives for revision that were formulated by the respondents and can be related to quality parameters:

To ensure that the text is coherent and logically composed;
To ensure that the text lives up to the standards of the organisation/company;
To check that the text or translation is correct;
To adapt the text to make it more suitable to its target group;
To adapt the text linguistically (amelioration);
To spar with, train or provide feedback to colleagues or freelancers (including translator). (Schjoldager et al. 2008: 803)

The second survey was carried out by Hernandez-Morin (2009a: 142), who asked professional revisers in France which of the following quality parameters they felt were crucial when revising a translation: accuracy, functional appropriateness, editorial quality, linguistic coding and other. The respondents ranked editorial quality first, followed by accuracy, linguistic coding and functional appropriateness.

In our own survey we chose to limit the number of quality parameters in our typology for two reasons. First, distinguishing between parameters of the same group, such as Mossop’s “tailoring” and “sub-language,” can be challenging, as noted by Künzli (2014: 10). Secondly, next to the typical translation quality parameters, we needed to include technical parameters specific to subtitling (see section 2.3). Our general translation parameters therefore are:

content and meaning transfer (including accuracy, completeness, logic);
grammar, spelling and punctuation;
readability (i.e. ease of comprehension and coherence between individual subtitles);
appropriateness (socio-cultural features of the audience).

2.3. Quality parameters in subtitling

The translation quality parameters described in the former section are also listed (albeit sometimes in different terms) in the Code of Good Subtitling Practice drawn up by Carroll and Ivarsson (1998).

Table 1

**Translation quality parameters used in the study and their counterparts in the Code of Good Subtitling Practice**

The Code of Good Subtitle Practice also lists a set of additional requirements specific to subtitling, and dealing with content, formatting and spotting, as shown in Table 2.

Table 2

**Additional requirements formulated in the Code of Good Subtitling Practice**

Carroll and Ivarsson’s code may seem outdated when one compares it with some subtitling practice today. Under the influence of digitisation and different formats and carriers, the advent of machine translation in subtitling and the impact of such other post-1998 trends as fansubbing as well as pleas for “abusive” subtitling (see e.g. Díaz Cintas 2010; Nornes 2004), subtitling practice is diversifying. However, this has not yet resulted in a new code of practice that is widely accepted (as Kuo 2014a confirms in her PhD) but rather in new trends the impact of which on professional subtitling varies widely and is hard to measure. The advantage of the 1998 code is that it is quite general and therefore widely applicable, especially since its basic tenets still hold. It is certainly the case that grammatical correctness, for instance, is upheld as a quality parameter by many broadcasters, especially by public broadcasters, such as the Flemish VRT, who increasingly rely on subtitles as a linguistic support for new immigrants. There is little doubt that correct grammar, spelling and adequate punctuation contribute to greater legibility and clarity, even if grammatically correct subtitles do not necessarily lead to higher quality or idiomaticity in terms of rendering colloquial speech. Similarly, many broadcasters take their deaf viewers into account in their interlingual open subtitling, as the Code also recommends under “content.” And while it is unavoidable that different interest groups have (slightly) different views on quality (e.g. subtitlers versus developers of MT for subtitling), it seems that there are only a few items in the Code that are not upheld to the extent that they used to: many subtitlers no longer care if their own spotting and reading speeds have certainly increased. (Díaz Cintas and Remael 2007) However, these shifts do not affect the overall validity of this research.

2.4. Translation revision procedures

Revision procedure is another key concept in the literature on revision. It does not answer the question of what has to be revised – this is related to revision parameters – but the question of how the translation is to be revised. Mossop explains: “You need to know not only what to look for, but how to look for it. To state the obvious, you cannot correct a mistake until you have found it.” (2007: 151)

Discussions on the most appropriate revision procedure usually centre on three issues:

Is the source text required for translation revision (bilingual revision versus monolingual revision)?
How often must the translation be read?
If the translation is read twice, for example, once with reference to the source text and once without, in which order must this be done?

In a number of publications resulting from Robert’s PhD research,[5] Robert (2012) and Robert and Van Waes (2014: 13-14) conclude that the answer to the first question is twofold. From a quality point of view, the bilingual procedure is significantly more efficient than the monolingual procedure, unless meaning transfer and content do not have to be checked; in that case, there is no significant difference in quality. As far as the process duration is concerned, the bilingual procedure does not take significantly more time than the monolingual one. Consequently, the bilingual procedure should be preferred to the monolingual procedure, unless the reviser is not asked to check the meaning transfer and content. In that case, the choice is free, since there is no difference in quality and no difference in time.

As far as the second question is concerned, if the reviser hesitates between the bilingual procedure (B) and one of the two two-step procedures (bilingual+monolingual and monolingual+bilingual), the answer is easy: the choice is free, since there are no significant differences at all with respect to quality and time

The answer to the third question is simple: the order in which the reviser operates is not important since research has shown that there is no significant difference between the two-step procedures.

Revision in subtitling will be dealt with below. To our knowledge, there is no research into the use of revision procedures in subtitling. We found only one reference to that aspect of the subtitling quality control process in the Code of Good Subtitling Practice, as indicated above. However, the code does not specify how the task should be performed.

3. Methodology

3.1. Online questionnaire as a research instrument

Kuo (2014a: 106) writes, “[q]uestionnaire surveys tend to be a popular research tool because they provide a relatively objective and efficient means of collecting information about people’s knowledge, beliefs, attitudes and behaviours (Oppenheim 1992; Sapsford 1999).” Besides, Kuo continues,

[wi]th the advancement of telecommunication technology and the prevalence of the Internet, online questionnaire surveys have gradually, to some extent, replaced traditional data gathering methods such as paper-and-pencil interviewing, mail surveys, telephone surveys and so on.
2014a: 106

Since our aim was to map the current quality control practices in subtitling, that is, to identify behaviours, we decided to use an Internet-mediated survey. This data collection instrument has many strengths, and also some weaknesses, as explained by Evans and Mathur (2005). The authors synthesise the vast literature related to online surveys and offer a number of recommendations. They also present the strengths and weaknesses of online surveys in a comprehensive, detailed and systematic manner (see Figure 2) and compare them with other survey formats. In addition, they offer a number of solutions for dealing with the major weaknesses of online surveys. In our methodology, we take these solutions on board.

Figure 2

**Reproduction of Figure 1 “The strengths and potential weaknesses of online surveys” (Evans and Mathur 2005: 197)**

3.2. Questionnaire design

The questionnaire survey was designed in Google Drive with the Google Forms function. The heading of the questionnaire, “Survey on professional subtitling practices,” was followed by a brief description of the aim of the research:

The aim of this survey is to map professional subtitling practises, focusing not only on the subtitlers’ profile, but also on their daily work. Results will be announced at the next Media For All 5 congress to be held in Dubrovnik, Croatia, on 25-27 September 2013.

The questionnaire consisted of 39 questions, grouped into three broad sections: (1) Profile (12 questions), (2) Own subtitling work (12 questions) and (3) Subtitling revision work (15 questions). Different formats were used for the questions: open/closed questions, dichotomous, multiple choice and rank order scaling. The questionnaire was pre-tested by two respondents (professional subtitlers) and subsequently adapted where necessary.

In the survey, we decided to concentrate on two aspects of quality assurance: (1) quality control, that is, the revision of the subtitles by the subtitler (“checking,” according to the standard, thus in the “translation stage”) and the revision of the subtitles by another subtitler (“revision,” that is, in the “post-translation stage” but before delivery), and (2) quality assurance aspects related to human resources (profile of the subtitlers) and technical resources (material available to carry out the subtitling task), namely, aspects of the pre-translation stage. In other words, we focused on the three phases of the translation process as described by Drugan (2013: 77-80) and summarised in Table 1 in the Appendix.

3.3. Questionnaire survey distribution

The questionnaire survey was posted at the beginning of July 2013 and remained on the Internet-based professional network LinkedIn for two months, in two groups related to the field: (1) “Subtitlers” (±2,400 members), and (2) “Subtitling is an art form” (±400 members). LinkedIn is a professional network with members from around the world. The first group, “Subtitlers,” was formed in order to promote high-quality subtitling and to maintain standards of professionalism within the industry. The second, “Subtitling is an art form,” is a group on subtitling, adaptation, translation: films, documentaries, cartoons, DVD/Blu-ray covers & booklets, scripts, etc. Consequently, these groups were considered to be representative of the profession and more specific than other translator groups with no specific focus on subtitling (e.g. professional translators and interpreters, ProZ.com). In addition, members of two associations related to Translation Studies were approached: the members of the European Society for Translation Studies (EST) via the EST Email Digest (Weeks 27-34/2013) and the members of the European Association for Studies in Screen Translation (ESIST). Roughly speaking, we believe that a population of approximately 2,000 respondents would potentially have been reached.

3.4. Data analysis procedures

The Google Forms function is user-friendly. After a form has been sent, Google Forms starts collecting the responses received. The user can choose how to store these responses, either in a spreadsheet or in forms only. We chose the former, since it allows for further analysis in Excel or SPSS. However, Google Forms also generates a summary of the responses, which is already very instructive and a good basis for discovering general trends.

4. Results

The results are presented in the order in which they appear in the translation process as illustrated in Figure 1: pre-translation stage, translation stage and post-translation stage.

4.1. Pre-translation stage

In the pre-translation stage, quality assurance was addressed through three different issues: (1) human resources (gender, age, country of residence, mother tongue, L2/L3 languages, professional status, type of activity, degree and professional experience), (2) planning, and (3) terminology and translation resources.

4.1.1. Human resources

In total we received 99 usable answers. The majority of the respondents were women (75%), aged between 23 and 64 (average 39, 2% < 25 years old, 53% between 25 and 40, 33% between 40 and 55, and 5% older than 55). The majority of the respondents were from Europe (86%), followed by North America (7%), South America (5%) and Asia (2%). In Europe, most respondents were residents of France (19), followed by The Netherlands (12), Finland (9) and the UK (8). All other countries were represented by a maximum of 6 respondents.

In total, the respondents had 21 different languages as their mother tongue. For the majority this was French, Dutch, Finnish and English, which is no surprise, since these are official languages in the most represented countries.

Only 10% of the respondents had English as their mother tongue; consequently, it is hardly surprising that the main source language was English, followed by French (see Figure 3). Many respondents (58%) also worked from a second (21%) or even third (6%) (or more) source language, as shown in Figure 4, and although there is considerable diversity, Spanish comes first, followed by French and German.

To identify the professional status of the respondents, we asked them to tick every answer that applied to them, namely, freelance, employee at an (audiovisual) translation agency, employee in a subtitling department, owner of a translation agency, and other. It was expected that some respondents might combine different statuses, and they did (see Figure 5): 9% of the respondents combined two jobs or more and these 9% all combined the status of freelancer with one or more functions. However, we must point out that 9% of the population only represents 9 respondents.

In the freelancer group, for 59% of the respondents, subtitling is the main activity (representing more than 75% of their activities). For 22%, it represented between 50% and 75% of their activities, and for 19% of the freelancers, this was less than 50%. For employees at a subtitling department, subtitling also represented at least 75% of their activities, as it did for employees in a translation agency (only 1 respondent reported that subtitling represented between 50% and 75% of his activities) and for translation agency owners (managers and CEOs). Among respondents who combined different statuses, more than 50% spent at least 50% of their time in subtitling, whereas regarding the type of activity performed, it seems that all respondents, whatever their status, combine at least two types, that is, subtitling, plus one additional activity (41%), or two (30%), three (13%), four (15%) or five (1%) additional activities.

Among these additional activities, translation was by far the most frequent, followed by subtitling revision, translation revision, teaching and other (see Figure 6). The “Other” 10% included translation for voice-over, transcription, captioning, dubbing, project management, reformatting, video production, subtitling and proofreading.

Figure 6

**Types of additional activity**

The questions relating to the subtitlers’ degrees revealed that 88% of the respondents had a Bachelor’s, that is, an undergraduate degree, a Master’s degree or graduate degree, or even a PhD, in a language-related field. The remaining 12% had a degree in other fields such as Law and Architecture, or did not have any degree. Among the participants with language-related degrees, 11% had a Bachelor in Translation or similar (Modern Languages, Applied Linguistics, etc.), 87% a Master’s and 2% a PhD. The majority of the language-related Master’s degrees were Masters in Translation, alone or in combination with another Master’s (67%). The holders of a Master’s degree in Linguistics and/or Literature (possibly in combination with another Master’s) constituted the second group, with 27%. Only 4% had a Master’s in Audiovisual Translation, but many programmes in Translation also offer courses in Audiovisual Translation. The 18% of the respondents who had a Master’s degree in Translation combined it with a Master’s in Interpreting, a Master’s in Linguistics and/or Literature, a Master’s in Audiovisual Translation or a Master’s in Teaching. Two respondents also had a third degree: a Master’s in Audiovisual Studies and a specialised degree in Audiovisual Adaptation. Finally, two respondents had a PhD in Translation Studies and in Film History.

In order to gauge the respondents’ professional experience, we asked them to quantify their experience in subtitling as well as in their other activities in years. Here we report on the figures for subtitling only: on average, the subtitlers had 10.37 years of experience (with a minimum of 0.5 year, and a maximum of 34 years).

To sum up, we can conclude that our respondents were mainly women, aged 39 (average), living in Europe, holders of a Master’s degree in Translation, often in combination with another Master’s degree, working as a freelancer, combining subtitling with at least one other activity, and subtitling mainly from English into their mother tongue.

In many respects, our results are similar to those of another survey conducted by Kuo (2014a), who received four times more answers (429). As in our study, the gender ratio of the respondents was close to 25% male and 75% female (Kuo 2014a: 138). The age of the respondents was also very similar: 53% aged between 25 and 40 in our study compared to 55.7% in Kuo’s study, and 33% aged between 40 and 55, compared to 34.5%. Regarding country of residence, The Netherlands, the UK, Finland, France, Spain and Denmark were most prominent in Kuo (first round survey, including subtitlers from all over the world), which also corresponds to our results. Accordingly, the main target languages in Kuo were Dutch, English, French and Spanish, again close to the languages indicated in our study, with English as the main source language in both studies: 81% and 83%. With regard to degrees, it seems that the proportion of respondents with a Master’s degree is higher in our study (77% versus 50%), whereas the proportion with a specialised qualification in subtitling is much lower in our study (6% versus 32.9%). However, it is probable that our respondents with a Master’s degree took Audiovisual Translation as part of their curriculum. Finally, it seems that our participants combined more activities: Kuo reported that 22.4% worked exclusively in subtitling, which is much more than in our study, where all the participants combined subtitling with one or more activities. On the other hand, their professional experience in subtitling was similar again: 85.1% of Kuo’s respondents had been working in subtitling for at least two years compared to 91% of ours. All in all, we can conclude that these results reinforce the reliability of our observations.

All these data relating to human resources are relevant to quality assurance in the pre-translation stage (see Figure 1), especially since they are also a factor taken up by the EN 15038, where it describes competences. According to the standard (European Committee for Standardization 2006: 7), professional translators should have at least five competences (translating competence; linguistic and textual competence in the SL and TL; research competence, information acquisition and processing; cultural competence; technical competence), which should be acquired through one or more of the following: formal education in translation (recognised degree), equivalent qualification in any other subject plus a minimum of two years of documented experience in translating, and/or at least five years of documented professional experience in translating. When we apply these requirements to our respondents, it seems that only three do not conform. Consequently, it seems that quality assurance at the level of human resources was taken seriously by all the stakeholders in the period covered.

4.1.2. Planning: translation brief and quality parameters

To analyse the planning aspects of the pre-translation stage and their relation to quality assurance, we focus on respondents who are suppliers, that is, on freelancers, and more specifically on freelancers who did not combine their subtitler status with another (n=80). They constitute the population that was targeted by the survey, although others also answered the questionnaire, as shown in section 4.1.1.

For Drugan (2013: 78) “planning” involves, among other things, “receiving and agreeing [on the] client brief”.’ The translation brief (TB) or instructions to the supplier of the translation is an important document since it determines the translator’s strategies. That is why we asked respondents how often they received clear instructions from their clients when asked to carry out a subtitling task (question 17), and how often the client explicitly asked them to focus on particular quality parameters (question 18), as listed in sections 2.2 and 2.3.

We observed that only 2.5% of the respondents never received clear instructions, 32% sometimes received clear instructions, 40% often did, and 22.5% always did. The 2.5% who answered “Other” explained that they usually had to ask the client explicitly, or that no specific instructions were given after initial training (clarifying each channel’s individual specifications). All in all, 62.2% always or often received clear instructions, which is a decent score, but not in line with quality assurance aspects of planning according to the EN 15038, which states that project management will include, among other things, “issuing instructions to all parties involved in the project” (European Committee for Standardization 2006: 9).

Table 3 illustrates the TBs received from a quality parameter perspective, indicating the frequency with which each quality parameter was explicitly mentioned by the client. The figures should be read as follows: for the parameter Content and transfer (column “Content”), 30% of the respondents reported that they never received instructions related to that parameter, 28% reported that they sometimes did, 13% reported that they often did, and 30% reported that they always did.

Table 3

**Translation brief parameters (in %)**

This indicates that the more often a parameter is explicitly mentioned in a brief, the more important it is for the client. However, our results show a great deal of variation. On average, approximately 32% of the respondents reported that translation quality parameters (mean percentage for content, grammar, readability and appropriateness) were never mentioned explicitly in the brief they received. This is a rather high percentage even if this does not automatically mean that these parameters are not important. We think that some clients may not mention them because they are thought to be part of any translation job. When we consider which parameters are mentioned at least sometimes (sometimes + often + always), we observe no real difference between the four parameters: 70, 69, 68, and 66%. The percentages are slightly different for the scores for “At least often”: grammar comes first with 49%, before readability (48%), content (43%) and appropriateness (35%).

As for the technical (subtitling) parameters, the results are markedly different: only 10% of the respondents reported that they were never asked to pay attention to the style guide. This is the lowest percentage of all the results for “Never.” This probably means that this is considered a very important parameter for the clients since it refers to their house style and is therefore explicitly mentioned in the brief. The same parameter gets 38% for “Sometimes,” 28% for “Often” and 25% for “Always.” This means that clients explicitly ask subtitlers to focus on this parameter at least sometimes in 90% of the cases, which is the highest score for all technical parameters. We also observe that the “At least sometimes” scores for all technical parameters are as high as or higher than the score for translation quality parameters (70% is the lowest score for formatting, but the highest for content). The first technical parameter in terms of frequency is style guide conformity, with 90% (at least sometimes score), followed by spotting (81%), speed (75%) and formatting (70%).

In conclusion, it can be said that clients tend to focus more on technical parameters than on translation quality parameters in their briefs, and that style guide conformity is a very frequent explicit requirement. These results are in line with the study by Kuo (2014a: 173), who reported that only 13% of the respondents never received any guidelines from their clients. In addition, 59% of her respondents also reported that they thought their main clients placed more emphasis on technical issues than on linguistic accuracy, compared with 27% for the opposite, and 14% for an equal emphasis on both linguistic and technical issues. (Kuo: 2014b)

4.1.3. Translation resources

For these aspects of the pre-translation stage and their relation to quality assurance we again focus on respondents who are suppliers, that is, on freelancers, and thus not on subtitling agencies. In her description of terminology and translation resources, Drugan (2013: 78) explains that these items involve, among other things, “preparing resources.” In the case of subtitling, translation resources can refer, in our view, to the availability of different resources or material. In the survey, respondents were asked how often the following material was made available: video file only, script only (= dialogue list), template only, or combinations of these. It should be noted that these resources are actually different features of what is called the source text in traditional translation. In section 2.3, the Code of Good Subtitling Practices states that “[s]ubtitlers must always work with a copy of the production and, if possible, a dialogue list and glossary of atypical words and special references” (Carroll and Ivarsson 1998) (our emphasis). In the survey, respondents were asked how often they received the video file, the script, the template or a combination of these source texts. The results are shown in Table 4.

Table 4

**Resources made available by the client (in %)**

A first important observation, in view of the recommendations of the Code of Good Subtitling Practices, is that a very high majority of the respondents reported that they never had to work with the script only (85%), the template only (86%), or a combination of the two (88%), which means that the video was generally available, alone or in combination with another type of source material. Whether the trend of working with the video material is due to the greater ease with which digital files can be transferred or to awareness of the need to use a video is difficult to ascertain: 20% of the respondents reported that they never had to work with the video file only, which means 80% do have to work with the video file only at least sometimes, but also that, in general, they worked with the video file and additional source material.

This is confirmed by the remainder of the results: 65% reported that they often worked with the video and script. This combination of source materials even obtained the highest score for “At least sometimes” (91%) and “At least often” (79%). On average, 14% of the respondents said that they at least sometimes had to work without the video, that is, with the script only, the template only, or a combination of these two. All in all, this is a rather positive result and, again, it is in line with Kuo’s study, where only 11% reported that they never received the video file. Only in so far as the availability of the template was concerned did the results differ slightly. However, the question was asked differently in Kuo’s questionnaire: the option of ticking a combination of resources was not given. As a result, 20% of Kuo’s respondents reported that they received a template at the following frequency rates: 22% “Often,” 20% “Sometimes,” 15% “Rarely” and 23% “Never.” Among our respondents, the template does not seem to be available very often, which means that this result needs further research or that the use of templates is in a state of flux.[6]

4.2. Translation stage

Quality assurance in the translation stage entails the checking of the subtitles by the subtitlers themselves, as required by the EN15038 standard. The respondents were asked to what extent they focused on the quality parameters discussed before (sections 2.2 and 2.3), independently of possible client instructions (question 22). In addition, they were asked how they generally proceeded when revising their own subtitling work.

4.2.1. Quality parameters

Table 5 below summarises the results for the quality parameters applied by subtitlers when revising their own translations. They are very different from the results for the quality parameters explicitly mentioned by clients in their brief. It seems that subtitlers focus on all the parameters almost always, with scores higher than 90% for all translation quality parameters except appropriateness (80%) and scores of at least 76% for all technical parameters. Here, our results seem to differ from Kuo’s: 84% of her respondents said that as far as subtitling quality was concerned, technical issues such as spotting were more important than linguistic accuracy, compared to 1% for the opposite and 15% for “equally important.” (Kuo, 2004b)

Table 5

**Subtitlers’ own quality parameters (in %)**

4.2.2. Self-revision procedure

As far as the revision procedure is concerned, the respondents were asked to describe their self-revision process by ticking off one or more operations from a list (question 24):

Revision of each subtitle immediately after translation
Revision of all subtitles of a scene after the scene has been subtitled
Once the whole video fragment has been subtitled, revision subtitle by subtitle without paying attention to the video
Once the whole video fragment has been subtitled, revision subtitle by subtitle while paying attention to the video
Once the whole video fragment has been subtitled, overall verification without the video
Once the whole video fragment has been subtitled, overall verification together with the video
Other

The results below are based on 76 answers, since 4 of out the 80 subtitlers ticked the “Other” option. In the analysis, we also decided not to take the very first option into account because it is difficult to distinguish this operation from the translation itself. Accordingly, we focused on the number of steps in the self-revision process and on the combinations of these steps (Figures 7, 8).

Furthermore, 45% of our respondents revised their own subtitling work in one step, and almost as many (41%) in two steps; 12% worked in three steps, and 1% in four or five steps. Consequently, we will report on the two most frequent procedures only, namely, self-revision in one and two steps.

When they self-revise in one step (Figure 7), almost all the revisers revise after having subtitled the entire video fragment (97%) and only a minority revise without the video (6%). In other words, they almost always revise with the video, either subtitle by subtitle (41%) or as part of an overall verification (50%).

When they self-revise in two steps (Figure 8), subtitlers apply diverse procedures. The majority (94%) start the self-revision process after they have subtitled the whole fragment (only 6% start with a scene-by-scene self-revision). In most cases (84%), they start by checking the fragment subtitle by subtitle, with or without video, and generally (87%) they finish the self-revision process with an overall verification with or without the video.

Figure 8

**Self-revision in two steps**

In conclusion, it seems that the subtitlers pay attention to all the quality parameters when revising their own subtitling work, independently of possible instructions from the client. Besides, they appear to revise in one or two steps, almost always relying on the video file at least.

4.3. Post-translation stage

For our study of the post-translation stage, we focused on the same aspects as for the pre-translation and translation stages. In other words, we investigated to what extent subtitlers receive clear instructions from their clients when asked to revise someone else’s subtitling work; we looked into the type of quality parameters on which the clients focused in their instructions as compared to the quality parameters on which the subtitlers focused independently of possible client instructions; we considered the material made available to the subtitlers and the revision procedure they applied while revising a colleague’s subtitles. The subtitlers were also asked how often they were requested to carry out this type of revision work, which means that the results below concern only those subtitlers who revise other subtitlers’ work, even if rarely.

4.3.1. Frequency of subtitling revision work

As many as 31 out of the 80 respondents reported that they never had to revise others’ subtitles, which means that the following results are based on the answers of 49 people.[7] Of these, as many as 40% revised others’ subtitling work less than once a month, which means that this activity is not a very common one. The details of our results are shown in Figure 9.

Figure 9

**Frequency of revision work among freelancers**

4.3.2. Client’s instructions and quality parameters

Surprisingly, the results for revision-related instructions are rather positive, although the scores are lower than the scores for clear instructions for subtitling work (see section 4.1.2): only 6% of the respondents reported that they never received clear instructions for the revision of someone else’s subtitling work; 54% sometimes did; 23% often did and 17% always did.

For the quality parameters on which subtitlers were explicitly asked to focus when revising colleagues’ subtitles, our observations are slightly different from those in the pre-translation stage, although style guide conformity is again a very frequent explicit requirement. Linguistic parameters seem to be more important than in the translation instructions (see Table 6): 48% of the respondents said that they were always asked explicitly to focus on content and accuracy, and as many as 53% claimed that they were always asked to focus on grammar. Scores for readability and appropriateness are as high as 43% for “Always.” All these percentages are higher than those for the technical issues, and higher than those in the pre-translation stage. However, no statistical test has been carried out, which means that we cannot claim that these differences are statistically significant – they merely indicate trends.

Table 6

**Quality parameters in instructions for revision (in%)**

Finally, just as in the translation stage, subtitlers were asked to what extent they focused on the listed quality parameters when they did not receive clear instructions from the client for a revising task. Again, the subtitlers seemed to focus on all the parameters when revising, just as they did when subtitling. The results are summarised in Table 7.

Table 7

**Subtitlers’ quality parameters for revision**

4.3.3. Material made available by the client

In “traditional” translation revision literature, the use of the source text for revision purposes is a central issue (see section 2.4). In subtitling, the “source text” consists of different components, as we pointed out in section 4.1.3. When looking into the material made available for revision by the client, we therefore took the same approach as for self-revision, asking the subtitlers how often the following material was made available: (1) subtitles + video file, (2) subtitles + script, (3) subtitles + template, (4) subtitles + video + script, (5) subtitles + video + template, (6) subtitles + video + script + template, or (7) subtitles + script + template. The results are shown in Table 8.

Table 8

**Material made available for revision**

A first and positive observation is that 58% of the respondents said that they always received the subtitles (ST) to revise together with the video file, and that 45% of the respondents also reported that they often received the subtitles to revise together with the video and the script. It was less common for them to receive the subtitles together with the video and the template, with or without the script: 45% and 49% report that this never happened. It was also rare for them to receive the script and the template only, without the video: 74% reported that this was never the case. The same trend can be observed for the frequency with which they received either script or template, without video: 53% and 66% reported that this was never the case. All in all, these results are rather positive, because clients tend to provide the necessary material when they ask subtitlers to revise another subtitler’s work.

4.3.4. Revision procedure

To conclude, our questions about revision procedures enquired how our respondents revised when they had the video file (question 36), when they did not have the video file but had the source text in another form (e.g. a template) (question 37) and when they did not have any source text (question 38, open question).

The results show that when they had the video file at their disposal, 49% proceeded in one step and 51% in two steps. Those who proceeded in one step almost always revised all the subtitles per scene with reference to the video (96%) and only rarely did so without reference to the video (4%). When they revised in two steps, a majority (67%) started with the revision of all the subtitles, per scene, with reference to the video and carried out a second revision with the video (38%) or without it (29%). A total of 29% started with a revision per scene without the video, followed by a second revision with the video, and a fourth but much smaller group did not use the video in any of the steps (4%). These results are illustrated in Figure 10.

Figure 10

**Revision procedure in two steps, video available**

The respondents were also asked how they proceeded when they did not have the video file at their disposal, but the source text in another form (e.g. a template): 42% answered that this never occurred. Of those who had to revise without the video, 46% proceeded in one step, and 54% in two steps.

Finally, our question enquiring into cases where subtitlers might be asked to carry out a revision without having any type of source text available received replies from 31 respondents only: 35% of them said that they always refused that kind of job and 26% said that they had no experience with such a work situation. The remaining 39% who accepted the job generally checked the spelling and grammar, or simply read the subtitles as a text.

5. Conclusions

This article on quality assurance in subtitling can offer only a snapshot of QA at a given time. Moreover, it is based on the results of an online survey, which means that we have reached only those subtitlers willing and able to spend time on replying to our questions and that we cannot fully control the veracity of their answers. In addition, the audiovisual translation sector is evolving quickly and data such as ours therefore tend to age fast. Having said that, however, many of our findings from 2013 are corroborated by the findings of Kuo (2014a), whose study covered some of the same issues. Moreover, our data also offer interesting insights into the current dynamics of the AVT sector that provide an excellent basis for gauging future developments.

Quality management, quality assurance and quality control, and especially (self-) revision as defined in sections 1 and 2 may not always be envisaged in exactly the same way by all scholars and practitioners, but they are central to most areas of translation practice and research today. Within the domain of audiovisual translation, research into quality parameters and procedures may be lagging behind, but in AVT practice “quality” certainly is a major issue. This is confirmed by our study, even if the focus of the quality procedures implemented by the subtitlers and their clients do not fully correspond to the procedures in other forms of commercial translation. The European standard for Translation Services EN 15038 may be followed less closely (see our comments on revision below) but then again, audiovisual texts and AVT have their own features, the most of important of which no doubt is the complexity of its multimodal source text, which has an impact on both translation and QA procedures.

Most striking about our data for “human resources” as a part of QA is the level of education and training of most subtitlers in our survey and the number of tasks they are able to perform, which does correspond to the demands of the European standard. Equally striking is the relatively high percentage of those engaged in subtitling as their main activity (more than 75% of their work) – even though the combination of subtitling with other professional activities such as translation, subtitling revision, translation revision or teaching is also very common (section 4.1.1). Similarly, with regard to the translation brief the subtitlers received from their clients, the results in terms of quality are more than acceptable (section 4.1.2) even if the subtitling clients seem to attach more importance to technical parameters than linguistic ones, differing in this respect from the subtitlers themselves, at least in our study. When one considers the subtitlers’ own quality concerns, which we investigated when enquiring about the quality parameters they applied when self-revising (section 4.2.1), they claim to take as many different parameters into account as possible (including content, grammar and readability), whereas their clients seem to focus more on technical than on linguistic parameters.

Regarding the availability of materials (section 4.1.3), 49% of the respondents indicated that they sometimes had to work with the video file only. This is a relatively high figure, considering that this situation is far from ideal, even though 65% reported that they often worked with the video and the script. The focus on style sheets rather than the linguistic rendering of the message and the failure to supply scripts seems to point to limited quality concerns among some clients.

Considering self-revision, we found that 45% of our respondents revised their own subtitling work in one step, and almost as many (41%) in two steps, whereas the revision procedures varied greatly. More than two steps were the exception. However, when it comes to revision, as many as 31 out of 80 respondents (39%) reported that they had never had to revise others’ subtitles, which shows that in this respect the subtitling sector follows neither the code of good practice nor the European standard. This sets it apart from commercial translation practice in other domains. It was therefore surprising to find that client revision instructions do exist and can be rather detailed (section 4.3.2), and also that they focus not only on style sheet matters but also on content and accuracy, and especially on grammar – even though the percentages for these aspects only hover around the 50% mark. Also in terms of material available for revision and revision procedures, the results were comparable to those for subtitling and self-revision respectively.

What does all of this signify? Whereas quality assurance and quality assurance procedures may not be a central topic in AVT research, quality as such has been a very hot topic at AVT conferences, such as the Media for All 5 conference at Dubrovnik in 2013. The reasons for this are manifold. The translation quality standards of highly qualified translators who set great store by the linguistic quality of their work are not fully shared by the industry (witness the already existing focus on technical parameters), which is feeling the pressure of globalisation in different ways. First, the economic crisis, and increased competition in the market, have resulted in falling subtitling fees. Secondly, that same economic crisis but also the increasing global demand for subtitling in an ever-increasing number of languages and formats is forcing the AVT industry to turn to translation technology for solutions offered by European Projects such as EU bridge (www.eu-bridge.eu) and SUMAT (www.sumat-project.eu). These are changing the profile of today’s subtitler – and in this sense the working environment of subtitlers seems to be evolving in the same direction as that of, say, technical translators – with post-editing MT becoming the new subtitling.

When considered from this angle, our findings explain the different perspectives of experienced and highly qualified subtitlers as opposed to their clients. They also offer interesting questions for research regarding the way in which the sector is evolving. The pressure to produce more subtitling in more languages has already led to the use of templates and is now prompting the sector to turn to MT for subtitling. Subtitling seems increasingly to be considered as a technical rather than an intellectual skill for which no university degree may be required. This begs the question to what extent the industry will continue to use qualified translators with Master’s degrees and also whether subtitlers will be required to do more multitasking and perform different professional activities. The latter development is already apparent from our data. Likewise, the increased use of technological solutions may lead to a consolidation of self-revision and revision into post-editing. Finally, how will university training cope with and evolve in the light of today’s quickly changing practices? We will revisit our questionnaire in five to ten years’ time in the light of such questions and compare the results.

Quality Control in the Subtitling Industry: An Exploratory Survey Study

Abstract

Résumé

1. Quality management, quality assurance and quality control