Is “quality” dead? Or can translators twist it creatively to find better paying customers?

You’d be hard pressed to find a translation provider who didn’t shout from the rooftops that they produce translations to the highest possible level of quality.

How about “We guarantee to deliver your work to a quality standard third from the bottom”?

Am I joking?

Well, perhaps not entirely…

I want to talk about “quality assurance warranties”. Would a bit of creative thinking help give “quality” a new twist and turn it into something that translation providers could actually use to find better paying customers? But would customers even understand what this is all about? Anyway, isn’t the whole question of defining quality an impossible “hot mess for the professional translation industry” [1].

In a recent post, I looked at the idea of how translation providers could find better paying customers by entering a bigger market. The internet age makes access to customers on a global scale a practical reality for solo translators and translation companies alike.

But once in a bigger market, providers need to narrow it down and attract just those customers who want to buy exactly what they can offer – at a price that’s going to provide a profitable margin. So the task is to differentiate – to stand out from the rest.

In this post we look at the role of “quality assurance warranties” – an explicit promise to deliver to a defined standard, and consider whether their use is a viable strategy for finding better paying customers.

Here’s the problem

Many translation buyers have difficulty figuring out what sort of translation provider is going to help them best. When is Google Translate an appropriate option? When is a competent freelancer the most cost-effective choice? Under what circumstances does a full-service, EN 15038-certified translation provider become relevant? What about the dozens of other options that fit in somewhere between Google and one of the super-huge agencies? What’s a buyer to do when every single translation provider in the world seems to claim they can deliver to the highest possible standard?

Some translation buyers, earnestly seeking to make a sensible purchasing decision, can hardly distinguish between what Google can offer them for free and what professional translators provide for a fee. When faced with a variety of professional providers – each with their own unique set of competencies – the task of choosing is often just too hard. The available options are to pick a provider at random or to buy solely on price. Either way, there’s a good chance the buyer will end up as another not-quite-satisfied customer.

Can we help buyers make more sensible choices?

Buyers with more experience tend to make more informed choices – but it takes time to learn the ropes and figure out what the important issues are. When talking to translation buyers, CSA found that companies with emerging translation and localisation requirements often start off badly. With experience they often end up with not just one single supplier, but a whole “stable” of language service providers and freelance translators as appropriate to meet diverse business needs:

“In the earliest or reactive phase of their work in localization, we find that localization managers either use the suppliers they inherited or hire the first ones who sound like they know what their doing. […] As they get more experience […] they will assemble a virtual team of LSPs and consultancies to cover desktop publishing, quality assurance, engineering, and translation projects.” [2]

Would it be possible to guide even inexperienced buyers to find and manage a “stable” of translation providers and help them make more sensible and cost-effective purchasing decisions – emulating the behaviour of more experienced buyers?

Inexperienced buyers might find it easier to make more appropriate choices if they could compare the different offerings of different providers across a small set of variables – simple enough so that they can actually make sense of them. These might include a simple measure of how reliable a provider might be, whether the provider has any special subject expertise which might be relevant, an easily understood indication of the sort of quality assurance capability the provider has, simple feedback on performance from previous buyers and a reliable deadline for delivery of the work.  Standardised information like this would help buyers make useful initial comparisons and influence how they evaluate the price offered by different providers.

Quality Assurance Capability

While translation providers all seem to promise to deliver to the highest possible quality standards, the reality is that different providers have different capabilities and deliver a wide variety of different levels of quality assurance.  This doesn’t mean that one approach to QA is necessarily better than any another; each may be appropriate for different projects, different clients and at different times – and at different prices. Different approaches to QA all have legitimate markets – in a market big enough, there are ready customers for every kind of “quality” – even the worst!

But how would buyers understand translation QA ?

Is it possible to convey the important issues of a big, complex (and controversial) topic to inexperienced buyers so they can usefully differentiate between different providers? Some of the bigger buyers will have localisation managers equipped with long experience or with university degrees in translation who know exactly what the issues are. But these are a tiny minority. For the average buyer, trying to decide between competing quotes, we might only have a few minutes (and more likely just a few seconds) to explain anything at all. Can we explain the meaning of the various provisions of EN 15038 or how this might differ from how competent freelancers might check their work?

Of course not!

To be useful, the essential meaning of what different providers are able to offer in terms of QA needs to be reduced down to something really simple that buyers with little insight into how foreign languages work can grasp intuitively and quickly.

A common way of approaching this problem is to represent complex ideas symbolically. Warranties and guarantees are often represented as seals and ribbons. Stars are another common symbol:

“Stars are often used as symbols for classification purposes. They are used by reviewers for ranking things such as movies, TV shows, restaurants, and hotels. For example, one to five stars is commonly employed to categorize hotels.” [3]

The Michelin system which is used to rate high quality restaurants is perhaps the most well-known star system. A single star denotes “a very good restaurant in its category”, two stars represents “excellent cooking, worth a detour”, and three stars, “exceptional cuisine, worth a special journey”.

It should be noted that “one star” doesn’t mean poor quality in this system – “one star” is already something of high value. But for many potential diners, the decision to go to a one or a three star restaurant will probably be determined by the state of their bank balance – the desire to eat at the most exceptional restaurants has to be weighed up against the cost.

A classification system using stars (or similar imagery) can potentially be used to symbolically represent different levels of QA capability. Buyers would not only quickly and intuitively figure that more stars is mean to imply greater value, but that this also implies that the service is likely to be more expensive. A buyer who can’t tell the difference between what Google Translate or a professional translator might give him, suddenly has something to go on…

Some, but not all buyers will ever want (or need) to understand the complexity which might lie behind warranties to deliver different levels of QA offered in the translation market – but to be meaningful and useful, the providers most certainly do.

“Acceptable quality” – what is it?

“Quality” in translation is often categorised as either good or bad. But we often forget that “acceptable quality” is a very broad category and can be different for different customers. Explaining the basics to buyers, CSA explains it this way:

“Acceptable quality for some might mean that there are no missing decimal points, while for another company, it might mean that the brand voice is rendered properly. And there are infinite points in between, including various quality-related standards. Help your suppliers understand what translation quality means to your business.” [4]

Producing different levels of “acceptable quality” implies the application of quite different approaches and human resources. Delivering more quality assurance usually requires a bigger investment in time and human resources. It makes intuitive sense that different levels of “acceptable quality” should vary in price to the buyer.

“Quality” and “Quality Assurance”

Translation quality, as Kirti Vashee puts it, is a “hot mess for the professional translation industry”. Confusion “is caused by the conflation of process quality standards with linguistic quality standards” [1].

No QA warranties can guarantee buyers that they will get “quality”, linguistic or otherwise. While existing quality standards typically “do not provide a buyer with a clear sense of the actual translation quality that will be delivered” [1] standards are nevertheless perceived by many to play an important role in the market. In a 2008 survey of buyer and vendor attitudes to quality issues, CSA report that

“… standards still matter to buyers more than LSPs think they do. For example, 58 percent of buyers believed that standards were “very important,” but only 42 percent of suppliers thought this to be the case…” [5]

The warranties proposed here (see the Appendix for the technical specifications) can only serve to assure the customer that the translation provider undertakes to carry out certain specified procedures to decrease the probability of error. (Was the spellcheck actually done? Did someone check the numbers? Did the agency actually select translators who have the qualifications specified in the warranty?)

For many buyers increasing the “quality” means reducing the chance of adverse outcomes. As Nataly Kelly of CSA puts it:

“For buyers, a lack of major errors often indicates that quality is acceptable. In industries where there is more potential for risk to end consumers – particularly where there is a risk of recalls – having a track record that is clear of major problems means that things are going well.” [6]

If the most useful working definition of “quality” is that it is “achieved when the buyer or the customer is satisfied” [1], then implementing specified QA procedures is likely to be just one of a number of factors which will determine customer satisfaction.

Issuing a warranty and promising to implement a specified level of quality assurance is one thing – actual performance is another! A system where providers warrant that they will implement a defined level of quality assurance is likely to be more meaningful to buyers if they can also see simple feedback from other customers on how they actually performed.

A buyer might well be wary of a provider who offers “5-star” quality assurance but who has poor performance ratings from previous customers. A “2-star” offering from a provider who has consistently good customer feedback (at half the price) might well be a more attractive proposition!

Giving customers a real choice

When competing providers can offer quality assurance at different levels this presents buyers with a real choice: when confronted with bids which vary by price and by level of QA, buyers get a sensible way to think about how much they might be willing to pay for what level of risk they are willing to live with. This is not a complicated equation – we all do it every time we go shopping.

How buyers resolve the trade-off between QA and price will be determined by such factors as the perceived importance of the project, the use the translation will be put to, the risk of adverse outcomes in the event of serious error – and the buyer’s budget.

For some, the cost of the most stringent QA might be too expensive. For others simply being prompted to taking QA into account in the decision-making process may provide the motivation for not always choosing the cheapest offer.

Isolating 5 different levels

Five different QA levels are proposed here (the technical specifications are detailed below). They were developed for use in the OpenBorder system of multi-factor tendering, but may also have some applicability outside this commercial system. The definitions are all drawn from the European translation standard EN 15038 and attempt to reflect the range and scope of quality assurance actually offered in the language services marketplace. The warranties align with how the provider marketplace is segmented in terms of the people involved in production and QA processes:

1. Students or newly qualified translators just entering the market and wanting to build their professional experience;
2. Experienced, independent translators who take sole responsibility for the QA of their own work;
3. Translation firms which take responsibility for translation work by implementing additional quality assurance and control measures (“revision”) after completion of the initial translation; or pairs of independent translators who cooperatively peer review each other’s work. This level implies that a minimum of 2 translation professionals are involved in the production of the work;
4. Translation firms which apply quality assurance processes to both the translation process and to production management. This level implies (at least) a three-man QA team which includes a translator, a reviser and a project manager.
5. Translation firms which have been independently audited by a certifying body for compliance with a recognised standard (EN 15038 and ISO 9001 in particular).

These five groupings of translation providers use different human resources and different techniques to implement different levels of quality assurance. All these groups are active in the market and compete with each other – but the buyer is often unaware of the different approaches to QA they take and may fail to find a reasonable explanation for significant differences in price due to the different QA methods employed.

From the providers’ perspective, there is no level playing field, and they often need to drop their price to compete with competitors who offer less.

Variations in the willingness to pay for different levels of quality assurance reflect different and legitimate market segments. Given that translation providers also vary in their capability to implement different levels of quality assurance, specifying up front what they are able (or prepared) to deliver allows them to be found by those customers who are willing to pay for precisely that level. This is competing on a more level playing field.

Compliance and policing the warranties

In a survey carried out by CSA in 2008, many buyers expressed some frustration that their translation providers “do not always use the resources they provide to them to facilitate translation, such as style guides and translation memory files” [6].

Negligently ignoring client-provided resources would be a clear violation of a warranty which specified their use. In such a case, the customer would have good cause to raise a dispute.

In the OpenBorder system, a simple, written contract between buyer and provider is generated for every transaction – large or small [7]. The contract incorporates the parties’ selected dispute resolution procedures, the buyer’s project specifications, the provider’s price, promised delivery time and the level of the QA the provider is willing to deliver.

OpenBorder holds full payment for the work on trust on behalf of the parties while it is in progress and the provider is not paid until any quality issues have been resolved in accordance with the agreed dispute resolution procedures. This system also protects the providers by guaranteeing them that if the terms of the contract are kept, they will be paid their full asking price within days after delivery.

OpenBorder has two systems to deal with providers who attempt to scam the system by offering warranties beyond their ability to deliver:

• Verification of Credentials
  In the event of a dispute, the onus will be on providers to verify the authenticity of the credentials implied in their warranty with documentary evidence (e.g. translator’s qualifications, the existence of a documented quality management system, EN 15038 certification etc as appropriate). Failure to adequately demonstrate the validity of their credentials to OpenBorder means that the dispute would automatically be settled in favour of the customer (i.e. with a completed refund of the provider’s asking price). The provider would also be excluded from further participation in OpenBorder’s online platform.
• Negative customer feedback
  OpenBorder’s buyers are prompted to provide simple feedback on the performance of their providers. This provides benefits to both parties – potential buyers can compare the provider’s promised performance with actual performance as rated by previous customers. Positive feedback gives providers the opportunity to build their online reputation and to expand their sales. Providers unable to keep their customers happy will find it increasingly difficult to do business –  irrespective of the level of QA warranty they are able to deliver.

Summary

Just as most of us do not read the fine-print in the warranties and guarantees we get with the consumer goods we buy, not all translation buyers will want to study the detail of QA warranties their providers may offer. But in the absence of an intimate understanding of what is involved in implementing quality assurance in translation, like the Michelin Guide, a simple 1, 2, 3, 4, 5 star system can help buyers to make sense of different pricing levels from different providers. Providers can use them to make the sale with potential customers who might otherwise have chosen a cheaper provider.

The translation of a scientific paper on computational chemistry done by a freelancer with a PhD in the subject and who offers a “2-star” warranty might sometimes be the most cost-effective option and may make the customer happier than one produced under the most stringent QA protocols by a “5-star”, EN 15038 certified translation company. On the other hand, a buyer who needs a complex set of manuals for a high-tech medical device translated into multiple languages and who wants to reduce the risk of error by having every sentence scrutinised by a whole team of professionals, might well want to choose a provider who is willing to offer a higher level QA warranty, and pay a premium for the additional peace of mind.

The detailed specifications for the proposed QA warranties are in the Appendix below.

Criticisms, ideas, suggestions, improvements or comments welcomed.

Paul

---

Appendix

Proposed technical specifications for 5 QA warranties

Language service providers may offer to deliver translation work to one of the following standards depending on their capability – all definitions have been drawn from the European Translation Standard EN 15038 [8].

For practical use, these specifications need to be expressed in two different ways:

• For buyers, as a symbol (e.g. stars) with a simple one-line definition (but with the ability to  click-through to a more detailed definition if required).
• For providers, these definitions could be translated into simple check-lists, providing a practical method for ensuring that all procedures promised to the client have been implemented prior to delivery.

Level 1 warranty: Apprentice

Providers offering a level 1 (“one-star”) warranty would include translation students or newly qualified translators who are keen to enter the market to begin accumulating professional experience. Under this warranty translators undertake to implement a minimum set of basic quality assurance and control procedures defined by European translation standard EN 15038, namely:

• Paragraph 5.4.1 (Translation)
  The provider warrants that terminology is appropriate for the specific domain (or that terminology and style guides supplied by the client have been used consistently), and that the text is fully compliant with the grammar of the target language.
• Paragraph 5.4.2 (Checking)
  The provider warrants that the work has actually been proofread, that it contains no omissions and is free of any grammatical, spelling or punctuation errors.
• Paragraph 5.4.6 (Final verification)
  A formal check against all the customer’s written specifications has been made to verify that the translation actually meets  the customer’s stated requirements.

Level 2 warranty:  Professional

Work supplied under this warranty might be from translation agencies who supply unrevised work or directly from independent, professional, freelance translators. In either case buyers will expect to pay more for work from an experienced professional than for a student or apprentice translator.

A warranty at the Professional level has two components: Firstly that the minimum set of basic quality assurance and control procedures (as per level 1) has been implemented. Secondly that the translation work has been carried by experienced, “professionally qualified translators” as defined in EN 15038, namely:

• Paragraph 3.2.2 (Professional competences of translators)
  Translators shall have:
  »» Undertaken formal higher education in translation (and have attained a recognised degree in translation); or
  »» Gained an equivalent tertiary qualification in any subject and has accumulated a minimum of two years of professional translation experience; or
  »» Has accumulated at least five years professional translation experience.

Level 3 warranty: Peer reviewed

Translations which carry a “Peer reviewed” warranty are produced by professionally qualified translators (as defined in levels 1 and 2 above) and then revised (edited, or peer-reviewed) by a second professionally qualified translator in accordance with the specific translation processes and quality assurance procedures defined in EN 15038.

Work supplied under this warranty requires the involvement of at least two professionally qualified translators. This implies significant additional production costs and buyers must expect to pay more for a reassurance that the work has had input from a second professional.  Peer reviewed warranties would typically be issued by translation companies or by pairs of independent translators who work cooperatively to review each other’s work.

Translation providers prepared to issue a “Peer reviewed” warranty, in addition to the requirements of  levels 1 & 2 above, undertake that their work is compliant with the following sections of the EN 15038 standard:

• Paragraph 3.2.3 (Professional competences of revisers)
  Revisers shall, at least, have the same minimum qualifications as translators (EN 15038 3.2.2) and are expected to have had some translation experience in the subject area of the particular translation.
• 5.4.3 (Revision)
  After the initial translation has been completed a reviser will compare the source and target texts and ensure that any necessary corrections have been carried out. The reviser shall be professionally competent in the source and target languages and have translation experience in the subject matter of the translation project.

Level 4 warranty: Quality management

In addition to the QA processes applied to the linguistic aspects of the production of a translation (as defined in the previous levels), translation providers also warrant that the work was produced under a quality management system which is compliant with the requirements of EN 15038.

In addition to the linguists, project managers are also subject to QA processes. The additional cost of purchasing from providers who offer a warranty at this level may become increasingly attractive to buyers as the complexity of their work extends beyond purely linguistic matters.

In addition to the requirements of the previous levels, under this warranty, translation providers undertake that the work is compliant with the following provisions of EN 15038:

• Paragraph 3.4 (Quality management system)
  The translation provider shall have a documented quality management system in place; and
• Paragraph 5.2 (Managing translation projects)
  The translation provider shall have documented QA procedures which govern how project managers handle the production process – this includes, inter alia,  the assignment of appropriate translators and revisers to projects, supervising timetables and taking responsibility for giving final clearance for delivery to the client.

Level 5: Certified

At this level, translation providers warrant that the work has been produced by an organisation which has current and valid certification under either European translation standard EN 15038 or ISO 9001.

Providers with ISO 9001 certification undertake that their production processes are fully compliant with the translation-specific requirements of the previous levels.

The certification process requires significant on-going investment in cash and human resources and providers must undergo an on-site inspection from the certifying body. Buyers must expect to pay a premium for work issued with this sort of warranty.

---

References:

[1] Kirti Vashee, (2010), Linking Translation Quality to Business Purpose http://kv-emptypages.blogspot.com/2010/04/linking-translation-quality-to-business.html

[2] DePalma, D. A., & Beninatto R. S., (2008), Localization Vendor Management Best Practices for Managing Language Service Providers, Common Sense Advisory, Inc., Lowell, Massachusetts, USA.

[3] The star classifications system – see http://en.wikipedia.org/wiki/Star_(classification)

[4] Nataly Kelly and Vijayalaxmi Hegde, (2011), Translation Quality: What LSPs Want Buyers to Know. www.commonsenseadvisory.com,  Common Sense Advisory, Inc., Lowell, Massachusetts, USA.

[5] Kelly, N., & DePalma, D. A. (2008) The Buyer-Supplier Quality Gap – How Customer and Language Supplier Views of Translation Quality Differ, Common Sense Advisory, Inc., Lowell, Massachusetts, USA.

[6] Kelly, N., (2008), Buyer Views of Translation Quality – What Every LSP Needs to Know, Common Sense Advisory, Inc., Lowell, Massachusetts, USA.

[7] By default, the EN 15038 standard (paragraph 4.4) requires an agreement between the translation provider and the buyer to record (at least) the commercial terms and service specifications. OpenBorder has adopted a strict interpretation of this sensible provision for all its participants to reduce the complexity and ambiguity which will inevitably arise as firms interact in an online, multi-lingual environment and to provide a mechanism for quick and easily managed dispute resolution.

[8] European Standard EN 15038:2006, Translation services – Service requirements, Approved by CEN 13 April 2006. (English version available from the British Standards Institution at http://www.bsi-global.com).

Related articles

• Is OpenBorder just another translation job board – or will it offer something fundamentally more valuable? (translationbiz.wordpress.com)
• Strategies for translation providers in an uncertain world… a survival guide. #xl8 #t9n (translationbiz.wordpress.com)
• Special invitation to translation providers (openborder.wordpress.com)