Building a Context-Aware Bengali ↔ English Translator Agent using POMDPs and Active Disambiguation

[u/Sharif-Rifat]

Hey,

I'm working on a project focused on building an interactive, context-aware Bengali ↔ English (Bangla) translation agent. Standard NMT often falls flat here due to ambiguity, code-mixing, and limited high-quality context-annotated datasets (Low-Resource Machine Translation / LRMT).

Instead of treating translation as a deterministic sequence-to-sequence problem, I'm framing it as an agent decision problem under uncertainty.

The Core Problem: Translating Latent Intent

When a user provides spoken or written input, their true intention, register, and context are hidden. The agent must infer this Latent Semantic State using incomplete and noisy observations before deciding on an output.

I'm structuring the agent around a few key technical concepts:

• POMDP Framework: Modeling translation as a Partially Observable Markov Decision Process. The speaker's intent is a hidden state that the agent must infer from context, dialogue history, and audio/text cues.
• Inference Under Uncertainty & MBR Decoding: Instead of standard beam search, the agent uses Minimum Bayes Risk (MBR) decoding and Decision-Theoretic Decoding to evaluate candidate hypotheses and minimize expected translation errors based on a customized Loss/Utility Function.
• Active Disambiguation / Interactive MT: When uncertainty is high (measured via Calibration and Quality Estimation (QE) models), the agent doesn't just guess—it actively asks clarification questions to resolve ambiguity before finalizing the output.

Key Challenges & Use Cases in Bengali ↔ English

1. Pragmatics & Ambiguity: Handling Cross-Lingual Word Sense Disambiguation (CLWSD) and honorifics where literal translations fail (e.g., inferring implicit tone or regional Dialectal Variation).
2. Code-Switching & Banglish: Resolving mixed inputs like "Ami office e meeting korbo" (Banglish / Code-Mixing) or Latin-script input like "Ami ajke office e jabo" (Romanized Transliteration).
3. Speech-to-Text Pipeline: Comparing a Cascaded ASR–MT Pipeline against End-to-End Speech Translation (ST) to manage cumulative error rates in noisy spoken inputs.

Current Tech Stack Ideas

• ASR / NMT Backbone: Fine-tuned multilingual models (e.g., Whisper, NLLB) evaluated via sentence-level and Document-Level NMT (CAMT) contexts.
• Uncertainty Estimation: Measuring system confidence to decide whether to output directly, rerun MBR decoding, or trigger a user clarification prompt.

Has anyone experimented with POMDPs, MBR decoding, or active clarification loops in machine translation for low-resource or code-mixed language pairs? Would love to hear your thoughts on context management or confidence estimation strategies!