Index-Translate: Meaning and Instructions, Together
For character dialogue, voice matters. For reader-facing explanations, tone and formatting matter. For internet memes, meaning often goes beyond the literal words. The same sentence can call for more than one good translation.
Index-Translate combines multilingual translation, instruction following, and meme understanding. Alongside the source text and target language, you can specify how the translation should read: preserve forms of address, use a more conversational tone, or return only the translation. The model uses these requirements to produce text suited to its intended use.
Teasing, irony, and cultural references require an understanding of why something is said that way. We include social and cultural context in our evaluations and assess translation instruction following separately, checking both the quality of the translation and whether the requested constraints were met.
Index-Translate is built upon Qwen 3.5 2B/9B/35A3B base (with 35B currently an in-training preview version) and trained through three stages:
Mid-train: At this stage, we adopt a standard WSD scheduler, mixing pre-training replay corpora, multilingual corpora, parallel corpora, and pivot corpora. We explored diverse data ratios, parallel alignment, and parallel organization methods, specifically increasing the proportion of pivot corpora during the annealing phase, enabling the model to learn cross-lingual semantic correspondences and expressions across diverse languages, laying a solid foundation for downstream translation tasks.
Post-train: Building on the multilingual foundation from mid-train, we trained three separate expert models—a general translation model, an instruction-following translation model, and a meme translation model—conducting SFT and RaR RL (Rubric as Reward) or RLVR for each.
Model-merge: Combining linear interpolation, MOPD, and Mix-RL, we consolidated multi-version capabilities into a unified model.
Post-Training Benchmark: Core Metrics
The main table summarizes post-training results for Index-Translate-35B (preview)/9B/2B and comparison models across FLORES-200, WMT24++, WMT26, instTrans, IFMTBench, the domain mean, MEME translation, and low-resource translation (FLORES_minor_pair). Index-Translate-35B (preview) achieves top results across FLORES-200 (0.8794), instTrans IFscore (0.8336), MEME (0.7405), and low-resource translation (0.8168); 9B significantly outperforms comparable model sizes; and 2B leads among lightweight models.
| Model | FLORES-200 COMET-22 ↑ |
WMT24++ COMET-22 ↑ |
WMT26 Judge ↑ |
instTrans IFscore ↑ |
IFMTBench IFscore ↑ |
Domain mean COMET-22 ↑ |
Meme MEME ↑ |
Low-resource COMET-22 ↑ |
|---|---|---|---|---|---|---|---|---|
| Index-Translate checkpoints | ||||||||
| Index-Translate-9B | 0.8789 | 0.8601 | 75.35 | 0.8209 | 0.8760 | 0.8451 | 0.7387 | 0.7992 |
| Index-Translate-2B | 0.8655 | 0.8489 | 60.26 | 0.7569 | 0.7584 | 0.8377 | 0.6443 | 0.7377 |
| Index-Translate-35B-A3B (preview) | 0.8794 | 0.8586 | 76.76 | 0.8336 | 0.8991 | 0.8438 | 0.7405 | 0.8168 |
| Comparison models | ||||||||
| Hy-MT2-1.8B | 0.8522 | 0.8401 | 49.35 | 0.4932 | 0.7161 | 0.8314 | 0.3643 | 0.3163 |
| Hy-MT2-7B | 0.8747 | 0.8593 | 60.51 | 0.6079 | 0.8741 | 0.8335 | 0.5139 | 0.4626 |
| Hy-MT2-30B-A3B | 0.8787 | 0.8624 | 66.81 | 0.6415 | 0.9029 | 0.8459 | 0.5812 | 0.6746 |
| translategemma-12b-it | 0.8732 | 0.8524 | 71.19 | 0.3068 | 0.2892 | 0.8347 | 0.4281 | 0.8021 |
| CohereLabs/North-Small-Translate-1.0 (218A25B) | 0.8784 | 0.8578 | 68.37 | 0.5294 | 0.8635 | 0.8357 | 0.6836 | 0.6855 |
| gpt-5.6-sol | 0.8650 | 0.8469 | 89.10 | 0.7624 | 0.9367 | 0.8311 | 0.7194 | 0.7669 |
| gemini-3.5-flash-lite | 0.8750 | 0.8497 | 79.52 | 0.6374 | 0.8854 | 0.8131 | 0.7034 | 0.8122 |
WMT26 is evaluated by GPT-5.6-Sol Judge across 20 target languages (0–100 scale). Domain mean is the unweighted average across five domain datasets. Low-resource is FLORES_minor_pair (COMET-22). Bold indicates best score in each column. ↑ Higher is better.
Low-Resource Translation and Instruction Following
FLORES_minor_pair covers 104,000 inputs across 1,040 translation directions among 62 languages. instTrans_minor contains 2,793 instruction-translation tasks from Chinese or English into low-resource languages; quality and IFscore are reported separately using GPT-5.6-Sol judgments. Off-target is the percentage of outputs in a language other than the target language; lower is better. Other scores use a 0–1 scale, with higher being better. Bold marks the best value in each column.
| Model | FLORES_minor COMET-22 ↑ |
FLORES_minor XCOMET-XXL ↑ |
FLORES_minor off-target ↓ |
instTrans_minor Quality ↑ |
instTrans_minor IFscore ↑ |
instTrans_minor off-target ↓ |
|---|---|---|---|---|---|---|
| Index-Translate-35B-A3B (preview) | 0.8168 | 0.7164 | 2.4% | 0.5151 | 0.7715 | 4.05% |
| Index-Translate-9B | 0.7992 | 0.6805 | 4.0% | 0.5222 | 0.7725 | 3.47% |
| Index-Translate-2B | 0.7377 | 0.4817 | 4.2% | 0.3050 | 0.6586 | 3.97% |
| Hy-MT2-1.8B | 0.3163 | 0.2014 | 54.2% | 0.0412 | 0.1313 | 65.27% |
| Hy-MT2-7B | 0.4626 | 0.3334 | 35.7% | 0.1121 | 0.2405 | 45.40% |
| Hy-MT2-30B-A3B | 0.6746 | 0.5359 | 14.5% | 0.2246 | 0.4449 | 15.47% |
| TranslateGemma-12B | 0.8021 | 0.6217 | 1.7% | 0.2412 | 0.2682 | 5.37% |
| North-Small-Translate (218B-A25B) | 0.6855 | 0.4919 | 13.2% | 0.2436 | 0.2902 | 26.89% |
| DeepSeek-V4.1-Flash | 0.8333 | 0.7297 | 1.3% | 0.4793 | 0.5854 | 5.73% |
| GPT-5.6-Sol | 0.7669 | 0.6918 | 10.4% | 0.5757 | 0.6866 | 7.73% |
| Gemini 3.5 Flash Lite | 0.8122 | 0.6995 | 3.3% | 0.3927 | 0.5584 | 12.18% |
Among the three Index-Translate models, 35B-A3B (preview) achieves the highest FLORES_minor_pair COMET-22 and XCOMET-XXL scores at 0.8168 / 0.7164, with an off-target rate of 2.4%. On instTrans_minor, Index-Translate-9B achieves the highest IFscore (0.7725) and lowest off-target rate (3.47%) among all compared models, with a translation quality score of 0.5222.
Full Index-Translate evaluation: seven categories and interactive radar chart ↘
Text & Images → Translation- Add text or image. Choose source and target languages, then enter text or upload an image to translate.
- Set your requirements. Specify tone, forms of address, glossary, or output format; custom instructions and vision translation are supported across 2B, 9B, and 35B.
- Read the translation. The translation appears progressively as it is generated, making it easy to read and inspect.
Index-Echo: Let Another Language Be Heard
Narration, dialogue, and commentary in video need to reach audiences through sound. Index-Echo takes source speech as input and generates the translation as target-language speech, letting the content be heard in a new language.
Index-Echo is an end-to-end S2S (Speech-to-Speech) translation model supporting speech translation and dubbing in 6 languages. With fast inference for dialogue and video clips, it makes the path from hearing the original to receiving translated speech more efficient.
S2TT: speech in, translated text out. An AuT encoder is connected to Index-Translate-2/9B, and the speech translation task is trained end-to-end, with aligned timestamps included in the translation output by default for synchronized sentence-level subtitle presentation.
S2ST: fully end-to-end speech-to-speech translation (speech in, translated speech out). Operating on a fully end-to-end architecture free from traditional ASR+MT+TTS cascading, S2ST removes the CosyVoice3 tokenizer and directly aligns the LLM's last hidden representations with CosyVoice3's semantic layer. Training proceeds in two stages: first freezing the backbone to distill an approximately 30M-parameter mapper for representation alignment, then unfreezing participating parameters for full-pipeline end-to-end joint training with DiffRO. Speaker voice information is extracted directly from source audio for zero-shot timbre preservation; it currently supports translation and dubbing from Chinese/English into 6 target languages.
Evaluation Highlights
In S2TT evaluation, Index-Echo-9B achieves an MT judge score of 0.857, ranking first in this comparison. Index-Echo-2B achieves a TS st_MAE of 0.395, the lowest timestamp deviation in this comparison (9B is second at 0.488).
The current online workflow starts with a Bilibili video and provides translated subtitles, proper-name and terminology settings, and a separate dubbing entry point. You can review the text before proceeding to dubbing.
Video → Subtitles and dubbing- Provide a video. Paste a Bilibili video URL, BV ID, or av ID, and select the target language.
- Review translated subtitles. Specify proper names and terms as needed, and export bilingual ASS subtitles.
- Continue to dubbing. Use the separate dubbing action to continue producing the video's voiceover.
Index-Homura: Translation That Fits the Rhythm of Dubbing
A line spoken in a few seconds can become much longer in another language. Cross-language dubbing therefore needs to do two things at once: preserve the meaning and fit the script into the original time window.
Index-Homura is a translation model with syllable control, designed for dubbing. You can set a target syllable-ratio range so that length becomes part of the translation task. The ratio is the number of translated syllables divided by the number of source syllables; the current online demo specifies length using a target syllable count.
Making length control a training objective. The HOMURA method uses GRPO reinforcement learning to combine a dynamic syllable-ratio reward with a translation-quality reward, jointly optimizing length constraints, meaning preservation, and fluency. The reward interval accounts for differences between target languages and the coarse granularity of counting syllables in short sentences, teaching the model to choose wording that fits the budget.
Evaluation Highlights
In the Sandglass evaluation, Index-Homura-9B achieves a composite score of 0.8660, ranking first on this leaderboard. The share of outputs with syllable deviation no greater than 10% is 81.92%, with a control slope of 0.968 (ideal: 1).
Dialogue and syllable count → Dubbing script- Enter Chinese dialogue. Choose English, Arabic, Spanish, or Japanese as the target language.
- Set the target syllable count. Use the slider or numeric input to specify the desired translation length.
- Review the script. Check its meaning and length before preparing the voiceover.
Index-Native Long: A Whole Novel, in One Pass
Long-document translation exposes clear limitations in translation-specific models. In this Chinese-to-English evaluation, Hy-MT2-7B drops on GuoFeng from 0.7334 at 4K to 0.0001 at 64K (and Hy-MT2-30B-A3B from 0.7535 to 0.0664), and on BWB from 0.7167 / 0.7301 to 0.0001 / 0.1103; by 16K, scores have already fallen to 0.0197 / 0.1615 and 0.0334 / 0.1432. Specialized translation capability does not naturally extend into robust long-document translation.
Even frontier models can face a serious bottleneck in long-form output. LongWriter (2024) observed that the frontier models evaluated at the time could accept long contexts yet struggled to generate sufficiently long text on request. Translating a whole novel requires sustained source coverage, consistent characters and plot, and a translation that reaches the end. A larger context window alone does not demonstrate these abilities.
Index-NaiLong targets long inputs and outputs and supports generating the complete translation of a novel in one pass. We make full-length translation a core task, examining output completeness, consistency of character names and forms of address, and continuity across passages, with the goal of producing a complete work that can be used further.
Evaluation Highlights
At 64K tokens, Index-NaiLong 9B scores 0.7382 and 0.7440 on GuoFeng and BWB Chinese-to-English translation, respectively, and 0.8514 on cross-domain books translated from English to Chinese; its 5-length overall means across the three corpora reach 0.7891, 0.7683, and 0.8848, ranking first across all three datasets with 274/274 full coverage. The full results report performance across all length tiers.
In the online demo, you can upload a document, read the translation as it is generated, and download the result. The current demo processes documents in segments and presents Chinese-to-English translations progressively.
Long document → Downloadable translation- Upload a document. Provide Chinese content in TXT or Markdown format.
- Read the translation in segments. The demo processes the document in chunks and displays English results as they become available.
- Download the translation. Save the result as Markdown or PDF for further use.
Evaluation
Detailed evaluations for Index-Translate, Index-Echo, Index-Homura, and Index-Native Long follow. Each model uses test sets and metrics appropriate to its task.
Index-Translate: Text & Multimodal Image Translation
We organize text translation evaluations across seven core dimensions: WMT, FLORES general translation, instruction following, low-resource translation, subtitle translation, MEME translation, and book translation (GuoFeng / OPUS-Books).
Among the 14 evaluated models, Index-Translate-35B (preview) ranks 1st in FLORES (0.87940) and instruction following (0.86635), and top-two in low-resource translation (0.81680) and MEME translation (0.74050); the 9B model ranks top-two in WMT, FLORES, subtitle translation, and book translation; the 2B model leads its size tier across categories.
| Dimension | Index-Translate-2B | Index-Translate-9B | Index-Translate-35B-A3B (preview) | Best Result / 14 Models |
|---|---|---|---|---|
| WMT (WMT24++) | 0.84890 | 0.86010 | 0.85860 | 2 / 14 (9B 2nd · Best 0.86240) |
| FLORES (FLORES-200) | 0.86550 | 0.87890 | 0.87940 | 1 / 14 (35B preview 1st) |
| Instruction following | 0.75765 | 0.84845 | 0.86635 | 1 / 14 (35B preview 1st) |
| Low-resource translation | 0.73770 | 0.79920 | 0.81680 | 2 / 14 (35B preview 2nd · Best 0.83330) |
| Subtitle translation | 0.82975 | 0.83515 | 0.83430 | 2 / 14 (9B 2nd · Best 0.83590) |
| MEME translation | 0.64430 | 0.73870 | 0.74050 | 2 / 14 (35B preview 2nd · Best 0.74240) |
| Book translation (GuoFeng / OPUS-Books) | 0.80950 | 0.81775 | 0.81520 | 2 / 14 (9B 2nd · Best 0.81820) |
Values are raw metric means; higher is better. All 7 dimensions include 14 evaluated models. Instruction following is the mean IFscore of instTrans and IFMTBench; Subtitle translation is the mean of OpenSubtitles and MuST-Cinema; Book translation is the mean of OPUS-Books and GuoFeng-Webnovel.
On individual benchmarks, Index-Translate-35B (preview) achieves an overall highest 0.8794 on FLORES-200, an instTrans IFscore of 0.8336, an overall highest 0.7405 on MEME, and 0.8168 on low-resource translation; the 9B model achieves top results on IFMTBench IFscore (0.8760) and subtitle benchmark MuST-Cinema (0.8979).
| Model | Raw mean | Relative score |
|---|
Scales are fixed using per-category normalization across all available models; changing the selection does not rescale the chart. Missing values remain gaps. “Best non-Index” uses the best baseline in each category and is not a single model.
Each category is independently normalized to 0–100 to show relative positions. Distances and areas do not represent raw score differences or overall capability.
Rankings reflect numerical order in the current data, without significance testing. For example, the top two subtitle-category means differ by 0.00065; this does not establish a statistically significant lead.
Download 12 raw benchmark scores · Download all seven-category means
Index-Echo: Speech Translation
Index-Echo evaluations cover both S2TT (Speech-to-Text) and S2ST (End-to-End Speech-to-Speech) tasks. S2TT evaluates translation semantics and subtitle timestamp alignment, while S2ST evaluates target-language content accuracy (WER/CER), translation quality (MT judge), and speaker voice preservation (cosine similarity).
S2TT Evaluation Results: Speech Input, Text Output
S2TT evaluation is based on an in-house video subtitle test set. Index-Echo-2B achieves the lowest TS st_MAE (0.395), with 9B second (0.488). For MT judge, Index-Echo-9B scores 0.857, ranking first among evaluated models; Index-Echo-2B scores 0.825, significantly outperforming open-source firered_audio (0.448).
| Model | MT judge ↑ | TS st_MAE ↓ |
|---|---|---|
| Qwen3.8-Omni-Flash | 0.887 | 1.004 |
| Index-Echo-9B | 0.857 | 0.488 |
| gemini-3.1-pro-think | 0.849 | 1.824 |
| Qwen3.8-LiveTranslate | 0.833 | — |
| Index-Echo-2B | 0.825 | 0.395 |
| gemini-2.5-flash-think | 0.817 | 2.402 |
| gemini-2.5-flash | 0.757 | 1.746 |
| Qwen3-Omni | 0.700 | 1.145 |
| firered_audio | 0.448 | 0.765 |
| SeamlessM4T-v2 | 0.060 | — |
Benchmark dataset: in-house video subtitle test set. ↓ Lower is better; ↑ higher is better. Bold indicates the best value in the column.
S2ST Evaluation Results: End-to-End Speech Translation & Dubbing
S2ST evaluation is based on eval36 (ZH→EN/ES/JA) and e2x (EN→ZH/ES/JA) across 6 translation directions, comparing end-to-end dubbing models (e2e) with cascade pipelines (Native Pipeline) at 2B and 9B scales. Japanese and Chinese use CER, while English and Spanish use WER; speaker voice preservation is measured by Speaker Cosine similarity to the source audio. Index-Echo end-to-end models outperform cascade pipelines on average content error rate across all 6 directions (9B: 0.0551 vs. 0.0596; 2B: 0.0554 vs. 0.0610), achieving notable gains in Spanish bidirectional, ZH→JA, and EN→ZH (where 2B e2e error rate drops by ~40%); voice preservation is fully on par (~0.72), demonstrating that hidden representation mapping does not degrade timbre.
| Model / Pipeline | Translation Quality MT judge ↑ |
Content Error Rate Content WER/CER ↓ |
Voice Preservation Speaker Cosine ↑ |
|---|---|---|---|
| Index-Echo-9B End-to-end S2ST (fd_rft) |
0.812 | 0.0551 | 0.722 |
| Native Pipeline (9B) ST LM 9B + CV3 direct read |
0.812 | 0.0596 | 0.719 |
| Index-Echo-2B End-to-end S2ST (sw_lam20) |
0.790 | 0.0554 | 0.720 |
| Native Pipeline (2B) ST LM 2B + CV3 direct read |
0.790 | 0.0610 | 0.719 |
| SeamlessM4T-v2 Open-source e2e baseline |
0.348 | — | <0.140 |
Benchmark dataset: eval36 and e2x across 6 translation directions (ZH→EN/ES/JA, EN→ZH/ES/JA). MT judge evaluates translation quality via gemini-3.1-pro (e2e and pipeline share identical ST LM translation texts); Content measures speech error rate (EN/ES use WER ↓, JA/ZH use CER ↓, mean after removing >0.3 outliers); Speaker measures cosine similarity to source voice ↑. Values are averages across all 6 directions. Bold indicates best score.
View detailed evaluation data across all 6 directions (ZH→EN/ES/JA, EN→ZH/ES/JA)
| Direction | Model / Pipeline | MT judge ↑ | Content (WER/CER mean) ↓ | Speaker Cosine ↑ |
|---|---|---|---|---|
| ZH→EN WER · eval36 n=100 |
Index-Echo-9B (e2e) | 0.840 | 0.0615 | 0.739 |
| Native Pipeline (9B) | 0.840 | 0.0477 | 0.718 | |
| Index-Echo-2B (e2e) | 0.830 | 0.0709 | 0.743 | |
| Native Pipeline (2B) | 0.830 | 0.0450 | 0.722 | |
| ZH→ES WER · eval36 n=200 |
Index-Echo-9B (e2e) | 0.793 | 0.0773 | 0.746 |
| Native Pipeline (9B) | 0.793 | 0.0916 | 0.746 | |
| Index-Echo-2B (e2e) | 0.728 | 0.0704 | 0.743 | |
| Native Pipeline (2B) | 0.728 | 0.0811 | 0.747 | |
| ZH→JA CER · eval36 n=200 |
Index-Echo-9B (e2e) | 0.815 | 0.0436 | 0.782 |
| Native Pipeline (9B) | 0.815 | 0.0552 | 0.778 | |
| Index-Echo-2B (e2e) | 0.760 | 0.0493 | 0.780 | |
| Native Pipeline (2B) | 0.760 | 0.0575 | 0.777 | |
| EN→ZH CER · e2x n=200 |
Index-Echo-9B (e2e) | 0.800 | 0.0485 | 0.623 |
| Native Pipeline (9B) | 0.800 | 0.0463 | 0.632 | |
| Index-Echo-2B (e2e) | 0.825 | 0.0372 | 0.623 | |
| Native Pipeline (2B) | 0.825 | 0.0614 | 0.630 | |
| EN→ES WER · e2x n=200 |
Index-Echo-9B (e2e) | 0.820 | 0.0660 | 0.729 |
| Native Pipeline (9B) | 0.820 | 0.0819 | 0.736 | |
| Index-Echo-2B (e2e) | 0.805 | 0.0702 | 0.735 | |
| Native Pipeline (2B) | 0.805 | 0.0867 | 0.732 | |
| EN→JA CER · e2x n=200 |
Index-Echo-9B (e2e) | 0.803 | 0.0337 | 0.710 |
| Native Pipeline (9B) | 0.803 | 0.0347 | 0.705 | |
| Index-Echo-2B (e2e) | 0.790 | 0.0345 | 0.698 | |
| Native Pipeline (2B) | 0.790 | 0.0342 | 0.707 |
Index-Homura: Syllable Control
In the Sandglass syllable-control evaluation, Index-Homura-9B ranks first among the 20 models and training variants on this leaderboard, with a composite score of 0.8660. Across 3,600 examples, 81.92% of outputs have a syllable deviation of no more than 10%. The control slope is 0.968, close to the ideal of 1; Index-Homura-2B also achieves a 63.08% deviation hit rate with a 0.947 control slope.
The evaluation covers 300 real subtitle lines, four target languages—English, Japanese, Arabic, and Spanish—and three length targets: short, natural, and long. The table compares Index-Homura-9B with all eight external baseline configurations.
| Model | Composite score ↑ | Syllable reward ↑ | Translation quality ↑ | Deviation ≤10% ↑ | Control slope ≈1 |
|---|---|---|---|---|---|
| Index-Homura-9B | 0.8660 | 0.9457 | 0.7863 | 81.92% | 0.968 |
| Index-Homura-2B | 0.7937 | 0.8259 | 0.7615 | 63.08% | 0.947 |
| Hy-MT2-7B | 0.6700 | 0.4839 | 0.8560 | 18.86% | 0.195 |
| Hy-MT2-30B-A3B | 0.6566 | 0.5577 | 0.7554 | 22.39% | 0.340 |
| Qwen3.5-9B | 0.5946 | 0.4708 | 0.7183 | 16.39% | 0.430 |
| Qwen3.5-35B-A3B | 0.5937 | 0.4011 | 0.7863 | 13.25% | 0.249 |
| Hunyuan-MT-7B | 0.5455 | 0.3722 | 0.7188 | 15.11% | 0.031 |
↑ Higher is better; the control slope should be as close to 1 as possible. Bold indicates the best value in each column. Each model is evaluated on 3,600 examples.
The composite score measures both length control and translation quality. Hy-MT2-7B scores 0.8560 on translation quality, above Index-Homura-9B at 0.7863. Index-Homura-9B achieves a syllable reward of 0.9457 and an 81.92% rate of deviation within 10%, reflecting the benefit of training for dubbing-length constraints.
Index-Homura-9B Results by Language
| Target language | Examples | Composite score ↑ | Translation quality ↑ | Deviation ≤10% ↑ |
|---|---|---|---|---|
| English | 900 | 0.8675 | 0.7950 | 78.67% |
| Japanese | 900 | 0.8941 | 0.8300 | 86.67% |
| Arabic | 900 | 0.8527 | 0.7661 | 79.89% |
| Spanish | 900 | 0.8496 | 0.7539 | 82.44% |
Index-Native Long: Long-Document Translation
This evaluation covers three frozen test sets (GuoFeng, BWB, and cross-domain books) across ZH→EN and EN→ZH for a total of 274 cases, spanning 4K–64K tokens across five length tiers. At 64K tokens, Index-NaiLong 9B achieves 0.7382 on GuoFeng and 0.7440 on BWB, and 0.8514 on cross-domain books. Its 5-length overall means reach 0.7891, 0.7683, and 0.8848, the highest among all evaluated models.
GuoFeng · ZH→EN
| Model | 4K | 8K | 16K | 32K | 64K | 5-Length Mean |
|---|---|---|---|---|---|---|
| Index-NaiLong 9B | 0.7836 | 0.8074 | 0.8080 | 0.8084 | 0.7382 | 0.7891 |
| Index-NaiLong 2B | 0.7704 | 0.7796 | 0.6508 | 0.6036 | 0.5460 | 0.6701 |
| Qwen3.8 Flash | 0.7974 | 0.7956 | 0.7572 | 0.6826 | 0.4939 | 0.7053 |
| GLM-5.3-Flash (Partial) | 0.7931 | 0.7957 | 0.7921 | 0.6018 | 0.5444 | 0.7054 |
| CohereLabs/North-Small-Translate-1.0 (218A25B) | 0.7952 | 0.7914 | 0.7309 | 0.5272 | 0.2353 | 0.6160 |
| Hy-MT2-7B | 0.7334 | 0.2872 | 0.0197 | 0.0006 | 0.0001 | 0.2082 |
| Hy-MT2-30B-A3B | 0.7535 | 0.4566 | 0.1615 | 0.0142 | 0.0664 | 0.2904 |
BWB · ZH→EN
| Model | 4K | 8K | 16K | 32K | 64K | 5-Length Mean |
|---|---|---|---|---|---|---|
| Index-NaiLong 9B | 0.7733 | 0.7695 | 0.7790 | 0.7756 | 0.7440 | 0.7683 |
| Index-NaiLong 2B | 0.7688 | 0.7669 | 0.7201 | 0.7100 | 0.6843 | 0.7300 |
| Qwen3.8 Flash | 0.7683 | 0.7676 | 0.7387 | 0.6262 | 0.5106 | 0.6823 |
| GLM-5.3-Flash (Partial) | 0.7632 | 0.7622 | 0.7107 | 0.6344 | 0.4748 | 0.6691 |
| CohereLabs/North-Small-Translate-1.0 (218A25B) | 0.7717 | 0.7704 | 0.7715 | 0.6077 | 0.2196 | 0.6282 |
| Hy-MT2-7B | 0.7167 | 0.2387 | 0.0334 | 0.0024 | 0.0001 | 0.1983 |
| Hy-MT2-30B-A3B | 0.7301 | 0.4408 | 0.1432 | 0.0746 | 0.1103 | 0.2998 |
Cross-domain Books · EN→ZH
| Model | 4K | 8K | 16K | 32K | 64K | 5-Length Mean |
|---|---|---|---|---|---|---|
| Index-NaiLong 9B | 0.9016 | 0.8771 | 0.8958 | 0.8980 | 0.8514 | 0.8848 |
| Index-NaiLong 2B | 0.8908 | 0.8736 | 0.8735 | 0.8435 | 0.8439 | 0.8651 |
| Qwen3.8 Flash | 0.8945 | 0.8910 | 0.8837 | 0.8582 | 0.5756 | 0.8206 |
| GLM-5.3-Flash (Partial) | 0.9045 | 0.8607 | 0.8974 | 0.8478 | 0.8871 | 0.8795† |
| CohereLabs/North-Small-Translate-1.0 (218A25B) | 0.8970 | 0.8952 | 0.8627 | 0.7873 | 0.7330 | 0.8350 |
| Hy-MT2-7B | 0.8246 | 0.3157 | 0.0335 | 0.0004 | 0.0057 | 0.2360 |
| Hy-MT2-30B-A3B | 0.8607 | 0.4395 | 0.1467 | 0.0097 | 0.1962 | 0.3306 |
Note: Evaluated using document-segale (max_size=8) pipeline. Bold indicates the highest score in each column. † indicates GLM is missing 7 cases in Books (267/274 cases scored across three corpora); mean is computed over scored subset only. Gemini 3.8 Flash is not included in the main comparison due to its official 65,536-token output limit repeatedly capping at 64K. North-Small scores at 32K/64K reflect experimental window expansion beyond its official 16K context limit.
Benchmarks
To support community research on real-world translation, complex constraints, and long-context generation, we officially open-source our evaluation benchmarks. Datasets and metadata are hosted in the 🤗 Index-Translate Benchmarks collection, with evaluation scripts, pinned data downloaders, and run instructions available on GitHub benchmarks/ ↗.
| Benchmark | Public release | Dataset | Evaluation code and guide |
|---|---|---|---|
| instTrans | 3,000 translation instruction tasks; 10 constraint types (format, terminology, tone, style, etc.) | 🤗 InstTrans-Bench ↗ | GitHub · insttrans ↗ |
| MEME | 3,638 Chinese-to-English cases; 703 terms and 857 senses | 🤗 Meme-Translation-Bench ↗ | GitHub · meme ↗ |
| SandGlass | 3,600 cases: 300 subtitles × 4 target languages × 3 length budgets | 🤗 Sandglass-Bench ↗ | GitHub · sandglass ↗ |
| NAtIveLong | Metadata for 84 BWB and 84 GuoFeng long-document cases, evaluation code and acquisition instructions | 🤗 NAtIveLong ↗ | GitHub · nativelong ↗ |
Note: NAtIveLong includes no source novels or reference translations: obtain official BWB data separately; GuoFeng provides a local reconstruction tool. The instTrans release does not include the 2,793 low-resource extension tasks in the evaluation tables. Original prompts are preserved; each directory's RELEASE_REVIEW.md records review changes and validation limits. Historical scores have not been rerun. Each benchmark has its own licensing terms; the repository's root license does not replace them. See full evaluation settings and IFMTBench preprocessing instructions in the GitHub repository.
Get started
The Index-Translate family includes four models. Choose your content type and desired output to open the corresponding demo or access open-source model weights. Official GGUF (llama.cpp), FP8, and NVFP4 (vLLM) quantized weights are now available in the Hugging Face Collection.
| Model | Variants | Core capabilities | Try it | Open-Source Models |
|---|---|---|---|---|
| Index-Translate | 2B / 9B / 35B-A3B (preview) | Text & multimodal image translation across 150 languages, instruction following, and meme understanding | Text / Image translation ↗ |
|
| Index-Echo | 2B / 9B | Fully end-to-end S2S, S2TT, and speech translation across 6 languages | Video speech translation ↗ | |
| Index-Homura | 2B / 9B | Control translation length using a target syllable-ratio range | Syllable-controlled translation ↗ | |
| Index-NaiLong | 2B / 9B | Long-form generation and complete translated novels in a single pass | Long-document translation ↗ |
Community
Join our official user & developer community to discuss model experiences, report feedback, and exchange prompt/glossary engineering tips. You can also scan the QR code below with QQ.
Index-Translate Official QQ Community
960123527Connect with the team and fellow developers to share insights, discuss deployment optimizations, test constrained translation, and receive the latest model updates.
Scan QR code to join
Featured Community Projects
Explore open-source tools and applications built by the community with Index-Translate: