Machine Translation - Neural MT

What is Machine Translation?

Machine translation (MT) is the automated translation of text or speech from one natural language to another. From early rule-based systems to modern neural approaches, MT has evolved from producing awkward word-by-word substitutions to generating fluent, contextually appropriate translations that rival human quality for many language pairs.

MT is one of the oldest AI applications—Warren Weaver proposed it in 1949—and remains one of the most commercially important. Google Translate serves over 500 million users daily, processing 100 billion words. The evolution of MT mirrors the evolution of AI itself: from hand-crafted rules to statistical learning to deep neural networks.

Approaches to Machine Translation

Rule-Based MT (RBMT) — 1950s-1990s

Pros: Predictable, explainable Cons: Enormously expensive to build, brittle, can't handle idioms well

Statistical MT (SMT) — 1990s-2015

Phrase-Based SMT

Neural MT (NMT) — 2014-present

The Transformer Revolution in MT

Attention-Based Translation

Translation Quality Metrics

Challenges in Machine Translation

Structural Divergence

Ambiguity Resolution

Low-Resource Languages

Modern MT Systems

Interview Questions

Q: Why did neural MT replace statistical MT? A: NMT handles the entire translation in a single model (end-to-end), while SMT required separate components (language model, translation model, reordering model) trained independently. NMT captures long-range context through attention, handles word order naturally, and produces more fluent output. NMT achieved +5-6 BLEU improvement over phrase-based SMT.

Q: What is the attention mechanism's role in translation? A: Attention creates a soft alignment between source and target words. For each target word being generated, attention computes which source words are most relevant and creates a weighted context vector. This solves the information bottleneck of fixed-size encoder vectors and enables the model to handle long sentences and word reordering.

Q: How do multilingual models handle 100+ languages? A: They share a single encoder-decoder across all languages, using a language tag to indicate the target. Shared multilingual representations emerge: similar concepts in different languages map to nearby vectors. This enables zero-shot translation between language pairs never seen together in training.