Summary: Word count is an insufficient metric for truly measuring the cost of modern translation (it fails to account for MT quality, string difficulty, etc.). Edit-effort estimation (EEE) aims to solve this issue by using modern AI models to predict translation effort so you can scope, route and price translation more accurately.
Word count has been the basis of localization budgets for years. But in an AI-first translation world, where more work is post-editing MT output than translating from scratch, that blunt metric is starting to crack.
The reason is simple: word counts don’t track effort.
Any translator can tell you that no two 100-word segments take the same time (or effort). Some are effortless while others are a small existential crisis.
MT engines handle short, repetitive UI strings in a high-resource pair like English–French surprisingly well. The human edit is often just a light polish. But feed it a dense, nuanced marketing paragraph into a tougher pair (say, a low-resource or morphologically rich target language), and those same 100 words might need to be rewritten from scratch.
In a world where “100 words” can mean anything from a 10-second tweak to a 10-minute rewrite, how do you estimate the real cost of translation at scale?
That’s where edit-effort estimation comes in.
Instead of treating every word as equal, we can use AI models to predict how much human work each segment is likely to need: no-touch, light edit, or heavy rewrite.
The argument in this article is straightforward: You will systematically misprice AI-driven translation if you keep budgeting on word counts alone.
Moving from word counts to effort counts is how you get saner budgets, fairer pricing, and better use of human expertise.
Machine Translation Quality Estimation (MTQE) and Edit-Effort Estimation: What’s the difference?
Under the hood, most of these systems are doing two closely related things:
- Estimating quality via methods like machine translation quality estimation (MTQE).
- Estimating effort using edit-effort predictions.
Quality estimation for machine translation (MTQE): Scoring MT without references
MTQE models take a source sentence and its MT output and estimate how good that translation is without needing a human reference. They’re trained on large datasets where linguists have already scored translations or annotated errors, so over time the model learns to recognize patterns that usually signal “this is fine” versus “this is risky.” However, results can vary significantly out of the box; models often require fine-tuning on specific client data to align with human quality expectations.
Machine translation quality evaluation vs effort: “Good” isn’t always “easy”
Edit-effort models use almost the same inputs, but they’re trained on a different type of label: how much work humans actually did on the output. Instead of just learning from quality scores, they learn from historical data, primarily Edit Distance (hTER), which measures how much the text had to change to reach professional quality.
While some models also factor in time or keystrokes, the industry standard relies on predicting the gap between the raw MT and the final human version.
The shift from encoders to LLMs
Historically, the industry has relied on encoder-only models(like BERT, XLM-RoBERTa, or COMET) for quality estimation. These are lightweight, fast, and extremely cheap to run, making them perfect for scoring millions of segments.
However, we are seeing a shift toward LLMs (decoder models) for this task. While LLMs are computationally heavier and more expensive, the cost of inference is dropping rapidly. The trade-off is often worth it. Unlike traditional encoders, LLMs can understand broader context, follow complex instructions (like style guides), and explain why a segment was flagged.
From that history, they learn to answer a slightly different question: not just “is this good?” but “how painful will this be to fix?” At run time, no linguist is sitting there scoring segments in advance; the model applies what it has learned and assigns each new MT segment to an effort band such as “no-touch,” “light edit,” or “heavy rewrite.”
In simple terms:
MTQE asks, “Is this any good?”
Edit-effort prediction asks, “How painful will this be to clean up?”
A note on current limitations!
While the theory behind QE is solid, the practical reality is often messier. Many off-the-shelf tools struggle to align with specific project requirements out of the box.
For complex domains or specific language pairs, “raw” QE scores can be unreliable, leading to results that are not yet usable without significant customization. It is an evolving technology, not a magic switch.
What the models actually look at
Instead of just counting errors, modern LLM-based systems look at a rich mix of signals to assess translation quality. For example:
- Grammar and fluency
- Whether the meaning is preserved
- Terminology and glossary compliance
- Consistency with translation memories
- Sometimes, even style-guide rules and readability features
Note: Traditional models focused exclusively on grammatical correctness, fluency, and meaning preservation. However, with the advent of LLMs and their growing usage, additional context can be fed into the model.
The final output is designed to be easily digested by a project manager (PM). As such, it is typically a simple color or probability. For example, traffic-light labels indicating high, medium, or light effort; a pass-fail flag; or a 0-to-1 score mapped to effort buckets.
Once those labels exist, they stop being abstract scores and start driving the workflow.
First, the TMS uses them to organize them into different processing paths. Then, inside the “human-needed” paths, it routes work to the right people.
1. Triage: Deciding what happens to each segment
High-quality or confidence segments might bypass human editing entirely or get only a quick skim. Medium ones go into a standard MTPE workflow. Low-quality ones are flagged for deep review or full retranslation.
2. Routing: Assigning work to the right expertise
Once a segment is in a human workflow, those same labels combine with domain and content-type signals.
- Low-quality legal content can go to a specialist legal team.
- Medium-quality technical documentation to a regular MTPE pool.
- Sensitive marketing strings to a named senior copy specialist.
The heavy, risky or high-value segments end up with the people best equipped to handle them.
Because all of this is structured, you can also layer dynamic pricing on top. Strings requiring low-to-no edits can be priced close to MT-only rates, while high-effort ones stay closer to full translation rates.
Instead of treating every word the same, rates follow the predicted effort.
Machine translation evaluation tools: How QE Scores drive real workflows
Different platforms implement this pattern in different ways, but the logic is similar.
- Traffic-light scoring. Tools like Smartling or Phrase use segment-level scores (high/medium/low) to gate what gets human attention. A lightweight model might decide which strings bypass editing, while a heavier MQM-style model is reserved for high-risk content.
- Pass/fail prediction. API-driven services such as ModelFront keep it simple: each segment is either “good enough to ship” or “needs a human.” This works well when you need a cheap, fast safety layer on top of MT.
- QE plus automatic post-editing. Some workflows (for example, Unbabel or TAUS-style setups) combine QE with LLM-based post-editing: QE decides which segments are clean, which can be auto-fixed by an LLM, and which must go to a human editor.
The names matter less than the pattern.
Transphere isn’t tied to any single vendor. What we care about is using these signals to sort the easy from the hard, then applying the right level of human expertise.
Whatever stack you use, the principle is the same: triage first, edit smartly, and let linguists spend their time where it actually moves the needle.
Where it works today (and where it doesn’t)
Effort estimation shines when conditions are favorable and stumbles when they’re not.
Where it’s a good fit
- High-resource language pairs. English-to-European language pairs (and a few others with ample data) tend to perform well. Models trained on them make more reliable calls about what will be “no-touch” versus “heavy lift.”
- Repetitive, controlled content. UI strings, support articles, product listings and internal documentation usually have a consistent structure and terminology. MT already performs well here, and effort estimation can confidently flag segments that require little or no human intervention.
- Large-scale, MT-heavy programs. When you’re pushing hundreds of thousands of words a month, it’s simply not realistic for humans to touch everything. Effort estimation helps you reserve human attention for the 20% that truly needs it, instead of spreading it thinly across 100%.
Where it’s not: When machine translation accuracy doesn’t match fluency
- Low-resource or typologically distant languages. For languages with limited training data or very different structures (e.g., highly inflected or agglutinative languages), predictions can be shaky. Models often underestimate the effort required, which is exactly how you end up with “green” segments that still need surgery.
- Specialized and high-stakes domains. Legal, medical, regulatory and safety-critical content carries strict terminology and real-world risk. Even when MT output looks fluent, it may still require careful subject-matter review. In these areas, QE and effort scores can support triage, but they should never be the only gate.
- Long-form and creative content. LLMs are prone to “normalization,” flattening distinct creative writing into generic, corporate prose. This makes it relatively unsuitable for marketing copy, brand storytelling, UX microcopy and anything heavily tone-driven. The model sees flawless grammar and predicts low effort; the linguist sees a boring translation that needs a full rewrite.
- Cross-sentence consistency. While LLM-first setups technically allow you to feed the model the whole document, due to cost and infrastructure constraints, more often than not, they will only see specific strings one at a time. This creates a blind spot for cohesion errors. A sentence might be grammatically perfect in isolation (scoring a “green” light) but use the wrong terminology compared to the previous paragraph. The model misses the forest for the trees.
- Calibration and cold starts. Off-the-shelf QE models often fail to align with specific client preferences immediately. Without a period of training or “calibration” on a client’s historical data, the scores may generate false positives, leading teams to distrust the signals initially.
- Self-bias. In modern workflows where an LLM translates the text and another LLM judges it, there is a risk of “grading your own homework.” If the estimation model shares the same underlying architecture or training data as the translation model, it may rate its translations higher simply because they align with its own probability distributions—even if they contain hallucinations or minor errors. This can lead to false positives, where the AI confidently approves its own mistakes.
The human factors you can’t automate away (yet)
Even when the language pair and domain are favorable, effort isn’t just about the text. It’s about who is doing the work.
- Translating into a native language vs. a second language can change effort dramatically.
- Two editors with different speeds, experience and subject-matter familiarity will not experience the same segment as equally “easy.”
- Word- or span-level highlights don’t always line up with what a human actually finds difficult in practice.
That’s why, at Transphere, we treat AI-based effort estimation as a guide, not a judge.
It’s a powerful triage layer, but you still need human review, domain expertise and feedback loops. Especially wherever the stakes, or the ambiguity, are high.
What changes for each stakeholder
For buyers and localization managers
Here, the biggest shift is how you decide where each unit of budget goes.
AI-based effort estimation empowers you to be more strategic with budgets. Instead of commissioning a full post-edit on every segment, you can prioritize spend, moving away from “full post-editing on everything” and towards strategic under- and over-investment.
High-quality segments can go straight to publication or a light review, ostensibly freeing funds for languages or content types that actually need linguist attention. This encourages effort-aware pricing—aligning rates with predicted cognitive and technical work rather than raw word counts.
Quality and effort signals also let you benchmark different MT engines or LLM prompts across languages and domains to see which tool performs best for your content. Ultimately, you’ll have more realistic ROI conversations with finance and more transparent discussions with your providers.
However, there is a potential trap here.
The temptation is often to pocket the savings from the “easy” content without reinvesting in the “hard” content. The real risk for buyers isn’t overpaying for easy segments anymore; it’s underfunding the high-risk pieces where a quality failure would genuinely hurt the brand.
For LSPs and in‑house localization teams
Effort estimation can really move the needle operationally, but it comes with inherent friction.
Ideally, it lets project managers route and prioritize jobs intelligently: senior linguists handle the tough projects; junior reviewers pick up the lighter tasks; straightforward “green” content flows through with minimal friction. That leads to smoother capacity planning, as you can staff projects based on predicted time and complexity rather than word count-related guesswork.
However, in practice, this is quite difficult (splitting a single file between “easy” and “hard” workflows is often technically messy). The real win here is usually in forecasting and pricing, ensuring you don’t burn out senior linguists on low-rate, high-effort clean-up jobs.
Many tools allow customization, meaning you can fine-tune the model with your client’s domain data to improve accuracy. But with that power comes responsibility:
- Monitor for drift and bias: Ensure the model isn’t “grading its own homework” too softly.
- Watch for misaligned incentives: If you pay linguists only for edits, you incentivize them to make unnecessary changes to “green” segments just to get paid.
- Ensure pricing remains fair: Automation ramps up efficiency, but it shouldn’t drive rates below a living wage.
Transphere’s view is simple: turning on a QE toggle is not a strategy. The differentiation comes from how you design the workflow, govern the thresholds, and communicate the impact to both clients and linguists.
For freelance linguists and reviewers
For freelance linguists and reviewers, this shift is both a risk and an opportunity.
On the positive side, well-calibrated effort estimation can protect you from unrealistic expectations. When segments are clearly labelled as “high effort,” it’s harder to justify blow-average rates. It also lets you focus your cognitive energy where it’s actually needed, solving hard linguistic problems instead of dealing with strings that were already fine.
But there is a genuine risk too.
If scores are opaque or badly calibrated, they can be used to squeeze rates while still expecting full-effort work. Worse, if payment models shift entirely to “pay-per-edit,” it devalues the critical work of verification. Reading a sentence and confirming its correctness takes time, even if no keys are pressed.
That’s why transparency matters: linguists should be able to see how segments were labelled, compare labels with their experience and challenge patterns that don’t make sense.
Your critical eye remains essential, but the role is shifting. You are moving from “translator” to “risk manager.”
Over-reliance on model scores can conceal errors, flatten nuance, and undervalue style. Positioning yourself as the expert who validates the machine’s work, certifying that the AI didn’t hallucinate or offend, is the strongest defense against devaluation.
The bigger picture: The impact of AI on machine translation quality
AI-based edit-effort estimation doesn’t herald the end of translators; it marks a shift toward effort-aware localization.
When you know ahead of time where MT is likely to shine or stumble, you can allocate budget and people far more intelligently. This is one layer in a broader AI/LLM stack for localization, alongside content triage, automated terminology extraction, and dynamic routing.
However, moving from objective word counts to probabilistic effort scores requires trust, and there are still open questions the industry must tackle:
- Data bias: How do we ensure fairness for low-resource languages where models are historically less accurate?
- Transparency: Can vendors increase transparency so linguists understand why a segment was scored a certain way?
- Economic sustainability: How do we ensure that paying for “effort” doesn’t inadvertently create a “race to the bottom” on rates?
What is clear is that doing nothing leaves you stuck with blunt instruments: word counts and fuzzy-match grids that might be predictable, but increasingly misrepresent the real work required in an AI-first world.
By experimenting thoughtfully (logging actual effort, calibrating thresholds, and keeping linguists in the feedback loop), you can achieve better quality, fairer pricing, and happier teams with the same budget.
At Transphere, we’re constantly updating our workflows to leverage state-of-the-art language models, improving translation quality at scale. A key part of that work is understanding where human expertise is non-negotiable, and where automation can safely take the lead.
Word counts tell you how much text you have. Effort counts tell you the true cost of getting it right.