Weight updates and harness search, alternating on one rollout budget. Neither half of the agent stays frozen, and both keep improving.
Whichever component you hold fixed is the one that caps the system.
A tool-using agent is not just a set of weights. Around them sits the harness: the code that decides which observations enter the context, how tools are exposed and invoked, how execution errors are handled, and when the interaction stops.
Consider a question-answering agent with a retrieval tool. Stronger weights cannot use evidence a brittle harness never retrieves; better retrieval cannot help a model that cannot synthesize what comes back. Whichever half you freeze becomes the ceiling.
Prior work adapts the two together, with the context side restricted to textual prompts. A prompt is one field of a harness. The rest of it, the tool schemas, the parsing, the turn budget, the retry policy, is where much of an agent’s behavior actually lives.
Optimizing the two together means coordinating two very different procedures. Weight updates are gradient steps in a continuous space; harness search is a proposal-and-accept loop over discrete programs, conditioned on many rollouts. They run on different timescales, and each reshapes the trajectory distribution the other optimizes against.
Running the phases concurrently either evaluates each update against a moving counterpart, confounding credit assignment, or requires synchronization barriers due to their mismatched update cadences. Alternation avoids this tradeoff: it decomposes the coupled problem into conditional updates, each performed with respect to a fixed counterpart.
Hover the steps, the arrows, the dashed lines, and the star for what each one means.
Train the weights under the current harness. Search a harness under the new model. Repeat.
Weight–Harness Alternating LEarning (WHALE) is a simple, modular algorithm for jointly optimizing a model and its harness.
Each cycle runs two phases. The weight-update phase trains the model with the harness frozen; the harness-search phase then proposes new harnesses with the model frozen, scores each on a fixed set of training examples, and carries forward the best-scoring harness it has seen so far — which may be one proposed several iterations earlier, or the harness the phase started from. Both operators are black boxes — swap in whatever you like.
Our instantiation uses online rejection-sampling fine-tuning for weights, training only on the model’s own verifier-accepted rollouts, and Meta-Harness for the harness, an agentic proposer that reads the source, scores and traces of every prior candidate.
How much of each to run per cycle — E epochs of weight updates and I
search iterations, the schedule (E, I) — is the one real hyperparameter, and it matters
more than we expected. Adaptive WHALE removes it: each phase runs a minimum length, then
stops once its own training signal — rollout reward for weights, best archive score for the harness
— has not improved for a fixed patience.
One fixed schedule, three domains, and the same answer each time.
The three domains are search question answering (SearchQA), mathematical reasoning with a Python interpreter (Math), and Chess Puzzles. Qwen3.5-2B drives SearchQA and Math, Qwen3.5-4B Chess Puzzles; the harness proposer is Claude Opus 4.7. Accuracy is mean@8.
Three baselines bracket it. Weight-only holds the harness at its starting version and trains the model alone; harness-only holds the model at its starting weights and searches the harness alone. Fast–Slow Training is the prompt-restricted control: the same two operators and the same schedule as WHALE, with the harness search confined to the system and user prompts. The gap to it is what searching the rest of the harness is worth.
With a single schedule, (E, I) = (0.6, 6), WHALE is the best system in every
domain — ahead of the stronger single-component baseline by
7.67 to 24.38 points, and ahead of FST by 4.15 to 13.00.
Which single-component baseline comes out ahead changes with the domain. Weight-only and harness-only finish effectively level in SearchQA, whereas weight-only is substantially stronger in Math and Chess Puzzles. WHALE improves on the stronger of the two under both orderings.
Hover a bar for what that method actually changed.
Either half can be the bottleneck, and moving one changes what the other can reach.
Drag across the plot, or focus it and use the arrow keys, to read every method at one rollout budget.
SearchQA is harness-dominant. Harness search alone matches the peak of weight-only training using 5.79% as many rollouts — 30,720 against 530,432. The model writes the query, but the harness decides how it is rewritten, how many documents come back, and how they are ranked. Retrieval accuracy climbs from 26.88% to 60.61% on harness search alone; weight updates barely move it.
Math is model-dominant. Harness search alone reaches 0.42% against 15.42% for weight updates. The bottleneck is that the base model reasons in prose until it hits the token limit and gets truncated — 95.83% of responses. Weight updates cut that to 30.83% by learning from its own rollouts that terminate in time. Harness-only tries the same thing from the outside — response caps, turn limits, a final-answer recovery step — but none of it overcomes the base model’s behavior.
The interesting part is what happens when you combine them. In Math, the first harness-search phase — after just one small weight update — raises format accuracy by 3.54 points using 4,608 rollouts. Harness search on the base model spent 46,080 rollouts to gain 0.63. A small weight update turned an ineffective search into an effective one.
Between too little evidence and too much commitment.
We ablate the alternation schedule (E, I) in SearchQA and Math, the harness-dominant and
model-dominant extremes of the previous section. Two things are compared against it: stagewise
optimization, which spends each component’s full budget in one pass, and a sweep over five fixed
(E, I) settings.
Stagewise is the over-optimized end. It is the common way to combine the two, with one
pass per component: train under the base harness with the full weight-only budget, then run harness search
under the resulting model for the full harness-only budget. WHALE with (0.6, 6) beats it by
5.32 points in SearchQA and 9.16 in Math, and passes its
final accuracy after only 29% and 49% of its rollouts. Each phase there optimizes against a counterpart
that then changes. The long weight-update phase fits the model to the base harness, so it adapts poorly to
whatever the search produces next; the long harness search fits the harness to the examples it is scored
on, and in Math it kept improving its training score across 60 iterations while gaining
0.21 points on the test set.
The sweep finds the noisy end. Shrink a phase too far and it decides on too little
evidence, so a noisy result is adopted as though it were a real improvement: with only two harness
proposal iterations per cycle, Math (0.2, 2) accepted a candidate whose score was inflated by
chance, adapted the model to it, and destabilized badly enough that we stopped the run. Growing the
budgets instead walks back toward the stagewise failure — scaling up from (0.2, 6)
never helps in either domain. Along (0.2, 6), (0.6, 6), (1.0, 10),
accuracy falls 50.09 → 48.34 → 45.93 in SearchQA and 28.33 → 24.79 → 24.79 in Math.
Adaptive WHALE removes the choice. A fixed schedule still has to be picked by hand;
the patience rule ends each phase on its own training signal instead. In SearchQA it reaches
52.82% — 4.48 points over the (0.6, 6) schedule of the main
comparison and 2.73 over the best hand-tuned schedule, (0.2, 6) — with 23% fewer
rollouts than (0.6, 6) needs for its own best. In Math it reaches
26.46%: 1.67 points over (0.6, 6) with 4% fewer rollouts, but 1.87
short of (0.2, 6).
| Run | Best mean@8 | Rollouts to best |
|---|
Hover a run in the plot or in the list; the other follows.
Six real rollouts. Same question, same model family — one pair fails, the other succeeds.
Below is one test item per domain that the base model under the base harness gets wrong and WHALE gets right, with both rollouts recorded verbatim.
Step with the ← → keys or press Play. The stage re-enacts what the harness shows the model and what the model does; the transcript below fills in verbatim as you go.
@misc{kim2026whalesimplerecipejoint,
title={WHALE: A Simple Recipe for Joint Harness-Weight Optimization},
author={Haechan Kim and Yoonho Lee and Gisang Lee and Chelsea Finn and Kangwook Lee},
year={2026},
eprint={2609.00196},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2609.00196},
}