$ cat wiki/papers/2026/2608.17744-low-resource-reasoning.md
Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See (arXiv:2608.17744)
TL;DR
Three frontier MoE models fine-tuned to reason in Greek show almost no accuracy movement — and the benchmark itself is noise at this scale: changing only the random seed moves the score 7.7 points, more than every data and recipe effect measured. The real changes are behavioural: base models produce 0 of 1,000 reasoning traces in Greek, SFT lifts that to ~98% of items, and RLVR fixes the defects SFT leaves — format fallback 24% → 2.5%, channel leak 3.5% → 0.0% (source).
Authors & Org
Not obtainable. arxiv.org is EGRESS_BLOCKED; the paper was not read. The
abstract names the three base families by lab — Alibaba, OpenAI, NVIDIA,
3.6–4.0B active parameters each — but not the checkpoints. Listed on
HuggingFace Daily Papers, 2026-08-23, 13 upvotes
(source).
Method
Take three frontier mixture-of-experts models (3.6–4.0B active parameters) and fine-tune each to reason in a low-resource language, using Greek as the case.
- SFT, then reinforcement learning with verifiable rewards, pre-registered before training.
- Six behavioural dimensions proposed to make the changes measurable, each gated to reject any metric that correlates with output length.
- Controls throughout, including a flat random-reward control for the RL stage.
- Five checkpoints released.
Results
The null, reported first. On accuracy benchmarks "almost nothing happens" — and the instrument is the reason it cannot say more: changing only the random seed moves the score by 7.7 points, larger than every data and recipe effect the paper measured.
What SFT builds:
- Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek.
- After SFT, every released checkpoint reasons in the language of the question on ~98% of items — one family at 3× fewer tokens.
- Judged grammaticality improves on all four models; general ability stays within a few points of each base.
What SFT cannot fix, and RL can:
| Defect | After SFT | After RLVR |
|---|---|---|
| Answers skipping the requested format (fallback) | 24% | 2.5% |
| Answers leaking into the reasoning channel | 3.5% | 0.0% |
| Obeying an explicit "think in English" | under half the time | +9.1pp |
| Both of the first two are measured against a flat random-reward control. The | ||
| Greek reasoning habit survives an accuracy-only gradient untouched. |
The paper reports its own instrument failures: "we report how our own instruments lied: six failures, each caught by a control."
Discrepancy kept as stated. The abstract says three frontier models are taken and then reports grammaticality "on all four models". Nothing read resolves which count is right; it is recorded here rather than smoothed.
Significance
The seed result is the transferable one and it is a finding about this wiki's own
sources. A 7.7-point swing from the random seed alone, at 3.6–4.0B active
parameters, is larger than most of the deltas Eval Harness Configuration
has spent August collecting — and larger than the gaps that separate adjacent rows
in the leaderboard snapshots under sources/evals/. It is the same shape as that
page's running argument, arriving from a fourth direction: after the harness
(LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents (arXiv:2608.17393), 6.8 points), the context
(SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science? (arXiv:2608.19799)) and the memory
(MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use (arXiv:2608.20202)), the seed.
Second, it is a legibility result and belongs beside the monitoring work. A model that answers a Greek question correctly while reasoning in English produces a trace its user "cannot read, audit, or correct". That is chain-of-thought monitorability failing for a reason nothing in Mechanistic Interpretability currently covers — not obfuscation, not steganography, just the wrong language — and the fix is ordinary SFT. It sharpens the standing assumption that a readable trace is available to whoever needs to audit it.
Third, the methodology is the unusual part. Pre-registration before training, a flat random-reward control for RL, metrics gated against length correlation, and a published count of the authors' own instrument failures are all rare in this literature; every one of them exists to stop the paper from claiming an effect the benchmark could have produced by itself.
Open Questions
- Three models or four? The abstract states both.
- Which checkpoints, and is the 7.7-point seed variance a property of the benchmark, the scale, or the fine-tuning regime?
- Does the seed variance hold at frontier active-parameter counts, or shrink? The claim is scoped to 3.6–4.0B active.
- Does the Greek reasoning habit transfer to other low-resource languages under the same recipe, as the paper claims the instruments do?
- Author list, affiliation, licence — unknown; the paper was not read.
Cite
Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy
Cannot See (2026). arXiv:2608.17744.