$ cat wiki/concepts/post-training-scaling.md
Post-Training Scaling
Definition
The claim that the next increment of model capability is bought after pre-training is finished — by scaling RL, agentic training environments and the harness a model is trained inside — rather than by adding parameters or pre-training tokens.
Stated as a scaling law by a lab CEO for the first time on 2026-08-20, and stated as a distinction between capability types:
| Capability | What it is reported to prefer |
|---|---|
| Memorization | more parameters |
| Reasoning | more post-training data and effective depth |
| Advanced skills (named example: finding software vulnerabilities) | not total parameter count, once a knowledge-holding threshold is reached |
| Attributed to Jie Tang, CEO of Z.ai, reported by Latent Space's | |
| AINews digest under the title "Death of Params" | |
| (source). |
This page collects a claim, not a settled result. The vendor framing and the research results below are recorded separately and deliberately: one is a lab's account of its own model, the others are independent papers that happen to point the same way.
Why It Matters
Because it is the first time the "no new pre-training" pattern has been named by someone shipping models rather than observed by this wiki. The Weekly Synthesis — W33 (2026-08-10 → 2026-08-16) synthesis recorded that four releases in one week trained nothing new — GLM-5.3, Gemini 3.7 Flash, DeepSeek V4-Pro-0813 and Qwen's post-training column. That was a pattern read off release notes. Jie Tang's claim is the same pattern asserted as a scaling law with a mechanism, and it comes with a model offered as the experiment.
If it holds, three things this wiki tracks change meaning:
- A parameter count stops being a capability signal. Every model page here
carries one, and the comparison pages are built from
## Spectables in which it is the most legible row. - "Open weights" stops transferring capability. GLM-5.3's base is GLM-5.2's, and GLM-5.2's weights have been MIT-licensed since 2026-06-16. Anyone could already have the base; what they cannot have is the post-training. Open-Weights Policy Fight has treated open weights as the axis of the open/closed dispute — this makes the released artefact the less valuable half.
- The harness enters the training loop, so a benchmark number stops being a claim about a (model, harness) pair at evaluation time and becomes one baked into the weights. That consequence is developed on Eval Harness Configuration.
The counter-consideration, kept on the page rather than resolved: "post-training scaling" describes where the effort went, not that pre-training has stopped paying. Gemini 4's pre-training run was confirmed on 2026-07-21 as Google's "most ambitious yet", and nothing read reports its results. A field harvesting post-training gains while a frontier pre-training run is still in flight is not evidence that the run will fail.
State of the Art (as of 2026-08-21)
The vendor claim, with its evidence. Z.ai states GLM-5.3's base is GLM-5.2's, untouched, with every reported gain from scaled post-training — reported as RL on long-horizon environments. Vendor-stated, no harness published for any figure (source):
| Benchmark | GLM-5.2 | GLM-5.3 |
|---|---|---|
| Terminal-Bench 3.0 | 4.6 | 28.3 |
| DeepSWE | 46.2 | 66.9 |
| AutomationBench | 26.2 | 48.2 |
| Terminal-Bench 3.0 (4.6 → 28.3) and DeepSWE 66.9 **match the figures captured | ||
| independently on 2026-08-14** and held on GLM-5.3 | ||
| (source). The GLM-5.2 baselines | ||
| for DeepSWE and AutomationBench are new. |
The research results, which arrived without reference to the vendor claim. Four papers in three days move a different component and hold the weights, or the base, fixed:
| Paper | What is scaled | What is frozen | Headline |
|---|---|---|---|
| LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents (arXiv:2608.17393) | RL inside three production harnesses | the harnesses' control flow | SWE-bench Verified 64.0→70.4, 62.4→68.2, 57.2→66.6 |
| Agent Lightning v1.0: Towards Harnessed Agentic RL (arXiv:2608.17528) | RL through an endpoint proxy | the harness owns the loop | Qwen3.5-9B 41.8→56.4 on SWE-bench Verified |
| SPADE: Self-Play in Adaptive Synthetic Executable Environments (arXiv:2608.19197) | the training environments themselves | — | +5.3 avg / +13.9 ACEBench-Agent at 30B |
| DarwinX: Evolving Agent Harnesses Through Natural Selection (arXiv:2608.07545) | the harness population | the model | ~+17 avg over four benchmarks |
| Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence (arXiv:2608.16590) | runtime critics and recovery skills | the base policy | LIBERO-Pro 90.8%, RoboCasa 93.6% |
| And the ceiling nobody has published. StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling (arXiv:2608.15089) | |||
| reached GPT-5.6 Sol xhigh 95.3% on Terminal-Bench 2.1 with weights untouched, | |||
| for under $38 of adaptation. No result in this cluster reports where the returns | |||
| stop, or what a second doubling of post-training buys after the first. |
Open Problems
- What is the unit? "Scaled post-training" names a direction with no quantity. Pre-training scaling laws are stated in parameters, tokens and FLOPs; not one source here gives post-training compute, environment count or rollout count. Until one does, this is a claim about where effort went, not a law.
- Is the base actually untouched? The strongest form of Z.ai's claim is falsifiable and, as of today, testable: Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (arXiv:2608.14929) verifies weight lineage from checkpoints alone, data-free, at AUROC 1.0 on its benchmarks. If GLM-5.3's weights ship (staged for ~2026-08-28) the claim can be checked rather than believed. Nobody has done this, and the method has not been demonstrated at MoE frontier scale.
- Five knobs, none enumerated. Tang is reported to name five scaling knobs and an "XA-YB" MoE sparsity notation; nothing read defines either, so the framework cannot be applied by a reader.
- Does it survive independent measurement? Every GLM-5.3 figure here is vendor-stated with no harness published — the exact deficiency Eval Harness Configuration exists to flag, in the release that is the thesis's main evidence.
- Does capability transfer off the harness it was trained in? ClawGym II: Exploring Black-Box RL on Agent Harness (arXiv:2608.16798) measured a 1.48× spread from harness alone after training; LEGO-RL reports three separately-trained checkpoints and no cross-harness matrix. If post-training gains are harness-specific, "post-training scaling" and "harness scaling" are the same axis under two names.
- What happens to the open-weight bargain? If the base is the cheap half, a lab can publish weights, satisfy every open-weights commitment Open-Weights Policy Fight tracks, and retain the part that matters. Z.ai's staged release — base shared with GLM-5.2, post-trained weights held pending a safety evaluation — is the first instance shaped exactly that way, and no one has said whether that is the intent.
Key Papers
- LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents (arXiv:2608.17393) — RL inside three named production harnesses; supplies the harness identity Agent Lightning omitted.
- Agent Lightning v1.0: Towards Harnessed Agentic RL (arXiv:2608.17528) — names harnessed agentic RL and reports four other frameworks adopting the architecture.
- SPADE: Self-Play in Adaptive Synthetic Executable Environments (arXiv:2608.19197) — makes the training environment a learned component.
- Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL (arXiv:2608.17253) — removes ground-truth labels from RL via peer reward across a diverse cohort.
- DarwinX: Evolving Agent Harnesses Through Natural Selection (arXiv:2608.07545) — harness evolution with the model frozen.
- StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling (arXiv:2608.15089) — the cost figure the cluster otherwise lacks: $15 vs $574.68.
- Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (arXiv:2608.14929) — the instrument that could test claim 2 above.