EvoHeir

Selection Is Not LearningChoosing Model Futures

Haoyi Yang* Kshitij Mishra* Nils Lukas Salem Lahlou
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi* Equal contribution
Paper PDF Code soon BibTeX
An illustrative ten-round lineage, not measured data. Each round explores two branches, screens one as the recipient, recovers experience into it, and keeps the result only if it beats the current model on fresh probes. Rejected rounds still leave their experience in the archive.

Self-evolving language models train on tasks they write for themselves, and every round must decide which updated model the next round inherits. Current loops decide by choosing tasks that look learnable, then inherit every update. We find that most of those updates do not help. HeirCommit judges the updated model itself on fresh questions. EvoHeir goes further and lets experience survive even when the model that produced it is rejected.

60 / 96Skill Self-Play updates on verified tasks that do not make correct answers more likely
68–83%fewer harmful commits with HeirCommit than random inheritance at the same rate
+5.3%relative math average over Agent0, the strongest baseline, across four backbones
45 / 48benchmark–backbone pairs where EvoHeir beats Agent0 after ten rounds

Task selection does not predict learning

Pass rate, disagreement, and frontier proximity describe the model before training. They say little about what an optimizer update will do to it. We call the mismatch the selection-to-update gap.

Most updates on verified tasks do not help

Correct-target NLL gain of 96 real Skill-SP updates on exact-valid tasks, sorted

Selected tasks look easier than they are

Support a target shows on the responses that selected it, minus its support on fresh responses (pp)

Under fresh responses, 52% of Agent0's and 42% of R-Zero's admitted tasks also fall off the native frontier. Whiskers show the reported interval for each estimate.

One EvoHeir round

EvoHeir gives exploration two routes into the next learner: the parameters it selects and the experience it absorbs. A useful experience donor no longer has to supply the next model's weights.

1 Explore native training from the root root St branch 1 branch 2 2 Screen paired gain on screening probes root b1 b2 recipient ties go to the root 3 Recover experience Et, Mt references attempts, no reference failed → repair set solved → keep set frozen teacher LoRA student repair keep (x, r, y<j) (x, y<j) (x, y<j) (x, y<j) Both copies start from the recipient's weights; the student matches the teacher token by token. S+ 4 Confirm frozen S+ root St fresh probes commit if v > 0 keep root v: paired gain of S+ over St Archive Mt keeps qualified responses from every branch whether or not the round commits, so a rejected round can still teach a later recipient.

1Explore

Train native branches from the current model, the root, exactly as the host loop would.

2Screen

Freeze the candidates and score each against the root on shared screening probes. The best one becomes the recipient.

3Recover

The recipient attempts current and archived problems without references. A frozen copy repairs its failures with a donor's reference and preserves its successes.

4Confirm

HeirCommit scores the final proposal against the root on held-out probes and commits it only if it wins. Otherwise the root carries on.

ℒrec(θ) = ⟨d(Tjrepair,Pθ,j)⟩ Drepair + λkeep ⟨d(Tjkeep,Pθ,j)⟩ Dkeep
Recovery objective. The student matches the repair teacher on failed attempts and the keep teacher on solved ones; d is the forward KL over the full vocabulary, with λkeep = 1.

HeirCommit cuts harmful commits on every host

We took 64 native updates from each of three self-evolution hosts and scored them on a pool kept apart from the decision probes. HeirCommit commits 23–52% of updates and keeps 78–87% of the useful ones.

Every update is one dot

The 64 native updates audited on each host. A loop that commits everything inherits every crimson dot.

Harmful share of commits

Share of committed updates that hurt the held-out pool. Matched random commits the same fraction at random.

HeirCommit turns negative pool gain positive on R-Zero and Skill-SP and raises Agent0's gain fivefold. It only needs the frozen successor and the root, so any host can apply it to a single native update.

EvoHeir leads on every backbone

Ten rounds of self-evolution on four backbones, three seeds each. EvoHeir has the highest math and general-reasoning averages on all four and beats Agent0 by 1.59–2.40 pp on the three base models. Of its components, the final confirmation step matters most: removing it costs 2.02 pp.

Math average after ten rounds (%)

Seven benchmarks weighted equally. ± is the SD over three seeds.

BackboneBaseR-ZeroAgent0No RecoveryEvoHeirOver Agent0
Bands: ±1 SD over 3 seeds for EvoHeir and Agent0
Change in math average from round 0 (pp). Each panel has its own y scale. R-Zero peaks between rounds 6 and 8 on three backbones and then slips; EvoHeir ends highest on all four. Hover a panel to read every round.

Gain over Agent0 on every benchmark

EvoHeir minus Agent0 at round 10 (pp, three-seed means). 45 of 48 cells are positive; the three negatives are 30-problem AIME sets.

−1+4 pp

Cite this work

@article{yang2026selection,
  title   = {Selection Is Not Learning: Choosing Model Futures},
  author  = {Yang, Haoyi and Mishra, Kshitij and Lukas, Nils and Lahlou, Salem},
  journal = {arXiv preprint arXiv:2610.22029},
  year    = {2026}
}