Review translations with AI

Bad translations can hurt your brand

Imagine visiting a website fully translated into your language, only to find that the translation is poorly executed. How would you feel as a user?

Or maybe a simpler example:

Would you hire me if I misspell your company name in my cover letter?

Poorly translated content can lead to a 60% drop in conversion rates.

That's why many of Lokalise's customers implement a translation review or QA stage in their localization process to enhance the quality of translations.

However, this process is not straightforward. For every issue identified, proofreaders must leave comments, log the error type, and provide corrections, which is very time-consuming.

As a result, reviewing translations takes an average of 39.5 hours per week 🤯

The research & design process

Step #1 - We interviewed 11 customers to identify the top problems

To better understand the nuances of customer review translations, we interviewed 11 customers and identified 3 issues.

Everyone hates spreadsheets

Customers are asking reviewers to fill in spreadsheets to log translation issues, but it's clearly ineffective and hard to organize.

Impossible to review all translations

For larger customers, reviewing over 10,000 translations weekly is unmanageable, even though they want to.

Different reviewers, different standards

Different reviewers have different standards, creating many false-positive issues and making it tough to measure quality accurately.

Step #2  - Design Sprint, a workshop to bring our ideas together

After conducting the research, I facilitated a design sprint session to brainstorm solutions together with my PM and Engineers.

Over a week, we:
• Explored the problem in-depth.
• Brainstormed and discussed multiple solutions.
• Developed ideas like creating a hands-off scoring function for L10n managers and assisting localization teams without language specialists.

HMW minimize the time for reviewers to spend on categorizing issues?
HMW...create a hands off scoring functionality so that L10n managers can set and forget?
HMW help localization team without language specialist to still perform LQA?

Ideas that we came up with from the workshop

Grab the workshop template on Miroverse

Grab the template

Close

Step #3 - Explore solutions

Based on the sketches and ideas generated during the workshop, I began prototyping various solutions and worked to align our internal teams.

Using comment panel as a way to leave suggestions

Highlight the issue area and label the issue

Using comment panel with quick access to add suggestions

A suggestion widget to tag issues

Initial solution — 
Simplifying Translation Tagging

An easy way to tag translation issues

Our initial solution aims to streamline the review process for proofreaders by enhancing their ability to identify and tag errors efficiently. While reviewing translations, proofreaders can quickly label any text as incorrect. They can also tag the specific issues associated with the error and provide additional comments for context.

The initial idea we had is to allow reviewers tag translation issues easily (but manually)

But humans are much lazier than we expected

“When users say they want a faster horse, they are actually asking for a car”

While testing with 7 customers, we found that although our initial solution managed to solve the inefficiency of using spreadsheets to log translation issues, and reviewers were generally happy with the feature, we learned that customers wanted something faster—they wanted a car, not just a faster horse.

Reviewing translations is crucial for them, but it has little impact on their actual work. Therefore, they want to minimize the cost of this process as much as possible.

We tested the prototype with ~7 customers to gather feedback

So we went to Warsaw for 3 days

To address this issue collectively, our team spent three days in Warsaw, where we reviewed all the feedback we had gathered. During this time, we organized a mini-hackathon, which provided us with the focused environment needed to develop and introduce an effective solution.

A hackathon day we had in Warsaw

And we introduced - 
AI, for reviewing translations.

Welcome to the AI-first, human-assist world.

This marks the industry's first-ever AI-driven tool where all translations are automatically reviewed by AI, significantly reducing the time proofreaders spend on manual checks. Proofreaders now only need to verify the AI's review to ensure it makes sense, streamlining the process and ushering in an AI-first, human-assisted era in translation review.

The actual Figma prototype (Might take a few seconds to load)

Feedback & Impact

📈 Adoption: 29% of enterprise team in Lokalise used the feature
🛍️ Retention: 19.1% monthly retention rate among teams.
🤩 Lead Generation: Attracted 804 Marketing Qualified Leads (MQLs) to try the feature.

Some feedback we received internally/externally

Curious more about what I learned? 
I wrote an article here.

Published in UXCollective · March, 2024

Designing better UX for AI — 8 Tips from 20+ companies

Let’s not add AI for the sake of AI. Let’s prioritize user needs, not the technology.

#AI

9 mins read | 10k+ views

Read next

Redesign a dashboard that drives 25% in sales