$ cat wiki/papers/2026/2608.14577-harmprofile.md
HarmProfile: Characterizing Harmful Distributions in Frontier LLMs (arXiv:2608.14577)
TL;DR
Treats harmful generation as an object of analysis rather than an attack outcome: 80,000+ validated harmful artifacts from 23 frontier LLMs across 13 model families, organised into 15 harm categories and 57 subcategories, and defines the resulting output distribution as a model-level risk profile. Its headline finding is that both harmfulness and diversity grow with model capability (source).
Authors & Org
Not obtainable. arxiv.org is EGRESS_BLOCKED from this environment and the
paper was not read; the HuggingFace Daily Papers snapshot carries title, id, date
and abstract only
(source).
Listed on HuggingFace Daily Papers, 2026-08-20, 19 upvotes — that community's
popularity signal and nothing more. The snapshot's Published column reads
2026-06-11 against an id in the 2608 block, so the listed date and the id
series disagree; both are recorded and neither is corrected
(source).
Code is stated at https://github.com/fresh-ma/HarmProfile.
Method
The stated premise is linguistic: just as behaviour can be characterised from an utterance corpus, model risk can be characterised from the content, severity and variation of its safety failures. So the unit of study is the harmful output itself, not whether an attack succeeded.
| Component | Figure |
|---|---|
| Validated artifacts | 80,000+ |
| Models | 23 frontier LLMs |
| Model families | 13 |
| Harm categories | 15 |
| Subcategories | 57 |
Results
- Frontier LLMs reliably produce harmful content at scale — stated as a property of the corpus, across all 23 models.
- Models exhibit distinct risk profiles: the distribution of what they produce when they misbehave differs between them.
- Both harmfulness and diversity grow with model capability.
- The paper's inference from that trend: frontier LLMs may appear safe yet harbour increasingly dangerous knowledge beneath the alignment surface.
What the abstract does not give: which 23 models, which 13 families, the capability measure that "grows with" is regressed against, any per-model figure, the validation procedure behind "validated", or how the artifacts were elicited.
Significance
It lands six days after a frontier lab said its instruments stopped discriminating, and it is the same problem approached from the opposite side. Anthropic's Risk Report: August 2026 raised catastrophic-misalignment risk in high-stakes settings from "very low" to "low" on the strength of increased uncertainty rather than a failed test, reporting that its most concrete task-based evaluations had saturated (source). A saturated pass/fail evaluation is exactly the instrument HarmProfile proposes replacing: a distribution has resolution where a threshold has none. Two models that both "fail safely" at the same rate can have entirely different profiles, and that difference survives saturation.
Whether it also survives its own headline claim is the harder question. If harmfulness and diversity both increase with capability, then a profile measured on today's models is a profile of today's elicitation methods, and the trend line is at least partly a statement about how much harder the strongest models are to exhaust. Nothing read separates the model produces more from we found more.
For AI Alignment this is a measurement-design contribution, not an alarm. The wiki holds several capability-scaling-versus-safety claims; this is the first that proposes an instrument whose output is a shape rather than a rate, and the first whose corpus is large enough to make per-family comparison possible at all. What it does not supply is any evidence about deployment: 80,000 artifacts collected under red-teaming conditions say nothing about what reaches a user, which is the claim it would be easiest to read into the abstract and is not in it.
Open Questions
- What is "capability" here? The central trend is a correlation against an unnamed capability measure. Parameter count, benchmark score and release date all correlate with each other, and the paper's conclusion changes depending which one it used.
- Elicitation confound — see above; "produces more harm" and "is easier to extract harm from at the effort we spent" are not distinguished in anything read.
- What does "validated" validate? 80,000 artifacts is far past human review at any plausible budget, so the validator is likely a model, and its error rate is unpublished. How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review (arXiv:2608.08975) is the standing reason this wiki asks.
- Distinct profiles — distinct how? No distance measure, clustering result or per-family figure was published in anything read, so "distinct" is unquantified.
- Publication risk. A public 80,000-artifact corpus of validated frontier-model harmful outputs is itself a dual-use artefact, and nothing read states an access control on it.
- Author list, affiliation, licence — unknown; the paper was not read.
Cite
HarmProfile: Characterizing Harmful Distributions in Frontier LLMs (2026).
arXiv:2608.14577.