Learn AI Create the Future — with Pranay
Main site

Multilingual and Indic NLP

Translating a low-resource language

A low-resource language has too little parallel text for a translation model to learn from directly, so it needs different tricks than English or Hindi do.

Read these first

On this page 5
  1. Why it exists
  2. How it works
  3. Where you have already seen it
  4. Remember this
  5. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

A low-resource language has too little parallel text for a model to learn from — too few matched sentence pairs.

Think about learning to cook a dish from a recipe with half the steps missing. You can guess at some of it from experience with similar dishes. Large gaps still leave you stuck, no matter how talented a cook you are.

Neural translation is the same. It learns from millions of matched sentence pairs. Give it a few thousand instead of millions, and no amount of clever architecture fully makes up the difference.

Why it exists

English, Hindi and a few other major languages have enormous amounts of translated text online: news, government documents, subtitles. Models trained on these languages have plenty to learn from.

Most of the world's roughly 7,000 languages have nowhere near that. Many Indian languages beyond the dozen or so most widely spoken have far less. Santali, Bodo, Kashmiri and others among the 22 scheduled languages have far less text online. Hundreds more languages, with no official status at all, have even less than that.

Low-resource translation is the set of techniques built specifically to squeeze usable quality out of that shortage.

How it works

  High-resource pair (English-Hindi):   millions of sentence pairs  ---> strong model

  Low-resource pair (English-Santali):  a few thousand pairs        ---> weak model,
                                                                          unless helped

The main help comes from not training that pair in isolation. Train the low-resource language alongside related, better-resourced ones, inside one shared multilingual model — the same idea behind XLM-RoBERTa. It then borrows patterns its own small dataset could never have taught it alone.

Where you have already seen it

  • Missing or noticeably worse translation quality for some Indian language pairs, compared to Hindi or Tamil.
  • AI4Bharat's IndicTrans2 project, built specifically to raise quality for India's lower-resource scheduled languages.
  • Community translation efforts — Wikipedia editors, local NGOs — manually building the parallel text that does not otherwise exist.

Remember this

  • Low-resource means too little parallel training text, not a property of the language itself.
  • Most languages in the world are low-resource for translation, including several official Indian languages.
  • The fix is rarely "collect more data alone." It usually means training the weak language together with related, stronger ones.

What to learn next

Developer — Code and libraries.

The clearest way to feel a low-resource shortage is to hit it directly. This reuses the tiny translator from how neural machine translation works and asks it to translate a word it was never trained on.

Setup

bash
pip install torch --index-url https://download.pytorch.org/whl/cpu

Asking a tiny model to translate an unseen word

low_resource_gap.py
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# The entire "language" this model has ever seen: ten words.
pairs = [
    ("one", "ek"), ("two", "do"), ("three", "teen"), ("four", "chaar"),
    ("five", "paanch"), ("six", "chhah"), ("seven", "saat"),
    ("eight", "aath"), ("nine", "nau"), ("ten", "das"),
]
SRC = sorted({w for w, _ in pairs})
src_vocab = {w: i + 2 for i, w in enumerate(SRC)}

# "eleven" was never part of training. This is a low-resource language
# in miniature: a word with zero examples for the model to learn from.
try:
    unseen = "eleven"
    idx = src_vocab[unseen]
except KeyError:
    print(f"'{unseen}' has no entry in the vocabulary at all.")
    print(f"The model cannot even represent this word, let alone translate it.")
    print(f"Vocabulary covers exactly: {sorted(src_vocab.keys())}")
Output
'eleven' has no entry in the vocabulary at all.
The model cannot even represent this word, let alone translate it.
Vocabulary covers exactly: ['eight', 'five', 'four', 'nine', 'one', 'seven', 'six', 'ten', 'three', 'two']

Line by line

This is not a bug — it is the honest shape of the low-resource problem. A word-level model has no fallback for a word outside its training vocabulary. Real systems reduce, but do not eliminate, this with subword tokenization: tokenization's "unbelievable" example still works for unseen words built from known pieces, but a Santali sentence made almost entirely of unfamiliar subwords faces the same wall in a smaller way.

Ten training pairs is an extreme, exaggerated version of a real problem. Some genuinely low-resource language pairs have training sets in the low thousands of sentences — still tiny by the standards of a model that ideally wants millions.

The failure is total, not partial. The model does not produce a poor guess for "eleven." It has no representation for the word to begin with. This is the practical difference between "low quality" and "no coverage."

Common mistakes

Assuming subword tokenization fully solves this. It shrinks the problem — rare whole words break into familiar pieces — but a low-resource language's subword pieces are themselves undertrained if the model saw little text in that language overall.

Collecting more monolingual text and expecting it to help translation directly. Translation needs parallel text — matched pairs across two languages. A pile of Santali news articles with no English counterpart cannot teach a model how the two languages relate.

Evaluating a low-resource checkpoint on the same test sets built for English. Quality benchmarks built around high-resource languages routinely overstate how well a technique generalises to a genuinely low-resource one — the exact caution this site's multilingual-nlp section opens with.

Try it yourself

Retrain the tiny model from the previous lesson with only the first three pairs instead of ten, then test it on the other seven. Compare how the failure looks — total KeyErrors across every dropped word — against training on all ten. That gap, seven working words versus zero, is the entire idea of "resource level" in miniature.

What to learn next

Researcher — Mathematics and papers.

Why parallel data scarcity is structural, not incidental

Parallel corpus size correlates strongly with a language's number of speakers online, its official/administrative status, and its historical digitisation investment — not with the number of speakers overall. Bhojpuri, spoken by tens of millions, has far less parallel training text than Icelandic, spoken by roughly 350,000, because of where digital text production has historically concentrated.

This makes "low-resource" a property of available data, not of the language's structure or the size of its speaker population — an important distinction, since it means the shortage is addressable by targeted data effort, not an inherent limitation of the language.

Mitigation techniques, roughly in order of how much they currently help

Back-translation (Sennrich, Haddow & Birch, 2016). Train a target-to-source model first, even a weak one, then use it to translate large amounts of monolingual target-language text back into the source language, manufacturing synthetic parallel data. This remains one of the highest-leverage techniques for genuinely low-resource pairs, since monolingual text is usually far more available than parallel text.

Multilingual joint training. Training a low-resource pair alongside typologically related, higher-resource languages inside one shared model — the mechanism explored for classification in cross-lingual transfer and for translation directly in NLLB-200 and IndicTrans2. Related languages, sharing vocabulary, grammar patterns or script, transfer more effectively than unrelated ones (Kunchukuttan & Bhattacharyya, 2020, on exploiting Indic language relatedness specifically).

Pivoting through a high-resource intermediate language. Translate low-resource-language-to-English-to-target, when no direct pair exists. This compounds error across two translation steps and is a fallback, not a preferred method, but it is often the only option when literally no direct model exists.

Transfer from a pretrained multilingual encoder. Initialising a low-resource model's encoder from a model like XLM-RoBERTa, pretrained broadly across many languages including the target, gives the translation task a head start even before task-specific fine-tuning begins.

Evaluation is itself harder here

Standard evaluation sets like FLORES-200 (NLLB Team et al., 2022) were built specifically because earlier benchmarks under-covered low-resource languages, making it impossible to even measure the gap being discussed. Reported BLEU and chrF scores on FLORES-200 consistently show a wide spread by resource level: high-resource pairs score highest, and low-resource pairs — particularly those with limited digitised text or non-standardised orthography — trail by a wide and well-documented margin, even inside models purpose-built to help them.

Key references

  • Sennrich, R., Haddow, B. & Birch, A. (2016). Improving Neural Machine Translation Models with Monolingual Data. arXiv:1511.06709 — back-translation.
  • Kunchukuttan, A. & Bhattacharyya, P. (2020). Utilizing Language Relatedness to Improve Machine Translation: A Case Study on Languages of the Indian Subcontinent. arXiv:2003.08925
  • NLLB Team et al. (2022). No Language Left Behind. arXiv:2207.04672 — introduces FLORES-200 and reports the resource-level quality spread directly.
  • Gala, J. et al. (2023). IndicTrans2. arXiv:2305.16307 — the Indian-language-specific version of this same effort.
  • Haddow, B. et al. (2022). Survey of Low-Resource Machine Translation. Computational Linguistics, 48(3). — a comprehensive overview of the techniques above and their measured effectiveness.

Current state and open problems

Massively multilingual models have measurably narrowed, but not closed, the low-resource gap. FLORES-200 results across NLLB-200 and IndicTrans2 both show real, usable quality for languages that had no viable open translation option a few years earlier, alongside a persistent quality gap against high-resource pairs that has not disappeared with scale alone.

Orthography remains a separate, under-discussed obstacle: several low-resource languages lack a single standardised writing system, so even collecting parallel text runs into which spelling convention to collect it in. No current technique resolves this — it is a data-standardisation problem, not a modelling one, and it sits upstream of every technique described above.

What to learn next