Review translations with AI Case Study

Bad translations can hurt your brand

Imagine visiting a website fully translated into your language, only to find that the translation is poorly executed. How would you feel as a user? Or maybe a simpler example: Would you hire me if I misspell your company name in my cover letter?

That's why many of Lokalise's customers implement a translation review or QA stage in their localization process to enhance the quality of translations.

Conversion impact

60%

drop in conversion rates from poorly translated content

Weekly time cost

39.5h

average time spent reviewing translations per week

However, the review process itself is not straightforward. For every issue identified, proofreaders must leave comments, log the error type, and provide corrections — which is very time-consuming.

The core problems

Everyone hates spreadsheets

Customers are asking reviewers to fill in spreadsheets to log translation issues, but it's clearly ineffective and hard to organize.

Impossible to review all translations

For larger customers, reviewing over 10,000 translations weekly is unmanageable, even though they want to.

Different reviewers, different standards

Different reviewers have different standards, creating many false-positive issues and making it tough to measure quality accurately.

1. We interviewed 11 customers to identify the top problems

To better understand the nuances of customer review translations, we interviewed 11 customers and identified 3 recurring issues.

Everyone hates spreadsheets

Customers are asking reviewers to fill in spreadsheets to log translation issues, but it's clearly ineffective and hard to organize.

Impossible to review all translations

For larger customers, reviewing over 10,000 translations weekly is unmanageable, even though they want to.

Different reviewers, different standards

Different reviewers have different standards, creating many false-positive issues and making it tough to measure quality accurately.

Interested in our research process? Tap to read

1. Internal Knowledge Gathering

I initiated the process by setting up a Miro board to map out the user journey. I invited input from Go-to-Market team members to ensure a wide range of perspectives. This collaborative approach helped us identify critical focus areas and integrate diverse insights, aligning our team from the start.

Internal Miro board used for knowledge gathering

2. Identifying the gaps and planning research

In collaboration with the Product Manager and User Researcher, we carefully crafted the research questions and developed interview scripts. This thorough preparation ensured that we could effectively gather deep insights during our interactions with users.

Research planning and interview script preparation

3. Collaborative Research

We engaged directly with our customers to map out their user journeys and identify their primary challenges. This collaborative research was instrumental in providing us with a detailed understanding of user issues, guiding our subsequent steps.

2. Design sprint — bringing our ideas together

After conducting the research, I facilitated a design sprint with my PM and engineers. Over a week, we explored the problem in-depth, brainstormed multiple solutions, and focused on three key questions:

How Might We

→Minimize the time for reviewers to spend on categorizing issues?
→Create a hands-off scoring functionality so that L10n managers can set and forget?
→Help localization teams without language specialists to still perform LQA?

Design sprint brainstorming workshop with PM and engineers — Ideas we came up with from the workshop

3. Explore solutions

Based on the sketches and ideas from the workshop, I began prototyping various solutions and worked to align internal teams. We explored four directions:

Using comment panel as a way to leave suggestions

Highlighting the issue area and labelling the issue

Using comment panel with quick access to add suggestions

A suggestion widget to tag issues

Initial solution — Simplifying Translation Tagging

An easy way to tag translation issues

Our initial solution aimed to streamline the review process for proofreaders by enhancing their ability to identify and tag errors efficiently. While reviewing translations, proofreaders can quickly label any text as incorrect, tag the specific issue type, and provide additional comments for context.

Animation showing how reviewers tag translation issues — The initial idea — allow reviewers to tag translation issues easily (but manually)

But humans are much lazier than we expected

While testing with 7 customers, we found that although our initial solution reduced the pain of using spreadsheets, it still was not enough. Customers did not just want a slightly better manual workflow. They wanted something much faster and more hands-off.

Reviewing translations is crucial for them, but it has little impact on their actual work. Therefore, they want to minimize the cost of this process as much as possible.

User testing sessions with ~7 customers gathering feedback on the prototype — We tested the prototype with ~7 customers to gather feedback

So we went to Warsaw for 3 days

To address this issue collectively, our team spent three days in Warsaw, where we reviewed all the feedback we had gathered. During this time, we organized a mini-hackathon, which provided us with the focused environment needed to develop and introduce an effective solution.

Team hackathon day in Warsaw — Hackathon working session in Warsaw

And we introduced — AI, for reviewing translations.

Welcome to the AI-first, human-assist world.

This marks the industry's first-ever AI-driven tool where all translations are automatically reviewed by AI, significantly reducing the time proofreaders spend on manual checks.

Proofreaders now only need to verify the AI's review to ensure it makes sense, streamlining the process and ushering in an AI-first, human-assisted era in translation review.

Embedded prototype of the AI-assisted translation review flow