Little Brother: a cost-aware coordination layer for parallel coding agents
Ash, Noah, Joshua · every number names the file in the repository it comes from.
1 The problem
Teams now run several coding agents at once, each in its own session with no view of the others. They duplicate work, retry paths a teammate already proved fail, and make changes that pass alone and break together. Git sees this only at merge time, and only when the text collides.
2 Architecture
Little Brother is a coordination layer: a Claude Code plugin on every laptop and one shared backend. Its design rule is cost: the expensive check runs only when the cheap ones can't decide.
- Claude Code hooks report every prompt and tool call
- Local normalisation on the laptop: touched files and symbols (TypeScript and Python), the repo's import graph
- Candidate pairs: other agents' recent work that overlaps
- Rules R1–R12 settle the clear cases in milliseconds
- A learned pair model settles most of the rest in well under a millisecond
- The LLM judge sees only what the model is unsure of, off the edit path
- A cost-aware intervention: deny, hint or nothing, depending on what acting is worth
- One alert per catch, delivered into the affected sessions
3 The classifier
Data
1,709 real labelled pairs: agent pull requests from AgenticFlict (each PR against the base branch's drift, merge-simulated) and STALE-bench Django pairs. 1,669 carry gold labels (ground truth from the merge or tests, weight 1); the rest are teacher labels from rules or the judge (weight 0.3). scripts/classifier/results/2026-10-24-real.md
Model
73 features per pair (paths, symbols, import-graph distance, intent overlap, failure memory). A gradient-boosted tree and a logistic regression are compared on out-of-fold predictions grouped by repository, so no repository is ever on both sides of a split; isotonic calibration and per-relation settle thresholds are fit on the same out-of-fold predictions.
Results
- Of the gold pairs no rule settles, the model settles 91% at 98% accuracy, so they never reach the LLM judge.
- Blocking precision 0.79 against 0.63 for rules alone.
- Calibration error (ECE) 0.06 → 0.02; 0.07 ms per pair against ~7 s for a judge call.
| Relation | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| overlap_compatible | 0.77 | 0.85 | 0.81 | 922 |
| textual_conflict | 0.79 | 0.71 | 0.75 | 710 |
| semantic_conflict | – | 0.00 | – | 37 |
| Architecture | Blocking precision | Blocking recall | Judge calls / pair | Latency / pair |
|---|---|---|---|---|
| Rules only | 0.63 | 0.95 | 0.00 | 0.05 ms |
| Pair model only | 0.79 | 0.68 | 0.00 | 0.07 ms |
| Cascade (rules → model → judge) | 0.63 | 0.95 | 0.03 | 193 ms |
Learning curve
The same grouped CV, training on 10–100% of each fold's rows (stratified, 3 seeds), always scored on the full held-out fold. Settling coverage reaches 91% by a quarter of the data; blocking precision rises to ~0.79 by half and is flat after that, so more of the same data will not move it: new kinds of labels will. scripts/classifier/results/2026-10-24-learning-curve.md
Limits
- No real positives yet for duplicate, stale or ordering pairs: those relations are never settled by the model and go to the judge.
- Semantic conflicts do not yet generalise across repositories (recall 0.00 out of fold on 37 pairs).
- Macro-F1 is 0.52 because of those empty classes; the model is used only where its thresholds were verified.
4 Evaluation
Protocol: Sonnet agents (Claude Code, headless) run the same task twice, Little Brother off and on, each on its own branches; a merge-and-fix step then merges the branches and gives one agent the chance to repair the result. Ground truth comes from git and the tests, never from an LLM. These are pilot runs: samples are small and every table names its run count.
| Metric | Definition |
|---|---|
| Precision | flags that matched a clash that really happened ÷ all flags |
| Recall | planted clashes that materialised and were flagged ÷ those that materialised |
| Pre-edit rate | catches that reached the later agent before its first edit on the clashing work |
| Interference (Δblind) | both tasks pass on their own branches, but a teammate's tests fail after a clean merge |
| Merged success | the merged branch passes every task's tests after the merge-and-fix step |
| Tokens and $ | new tokens (input + output + cache writes) and Claude Code's cost estimate per run, fix step included |
Larger tasks pilot · 18 runs
Multi-file features with a planted clash; nine off/on pairs. apps/evals/results/pilot-1024/large/report.md
| Little Brother off | Little Brother on | |
|---|---|---|
| New tokens / run | 42.3k | 35.1k |
| $ / run | $0.35 | $0.26 |
| Final merged success | 67% | 89% |
| Runs with a merge conflict | 67% | 11% |
| Detection precision / recall | – | 100% / 100% |
| Caught before the first edit | – | 89% |
Planted suite pilot · 60 runs
Two agents per case with a deliberate duplicate, conflict or stale path; pointed and natural prompts. apps/evals/results/pilot-1024/planted/report.md
| Little Brother off | Little Brother on | |
|---|---|---|
| New tokens / run | 28.0k | 27.7k |
| $ / run | $0.18 | $0.18 |
| Final merged success | 93% | 100% |
| Runs with a merge conflict | 27% | 3% |
| Detection precision / recall | – | 100% / 64% |
| Caught before the first edit | – | 88% |
| Median time to detect | – | 20.4 s |
STALE-bench, constructed interface changes pilot · 64 runs
One agent changes an interface another reuses; two passes of 16 off/on pairs. apps/evals/results/pilot-1024/stale/report.md
| Little Brother off | Little Brother on | |
|---|---|---|
| Interference (passes alone, breaks merged), pass 1 | 50% | 19% |
| Interference, pass 2 | 100% | 44% |
| Own task solved alone, pass 1 / 2 | 88% / 94% | 72% / 66% |
| New tokens / run, pass 1 / 2 | 20.2k / 22.9k | 21.6k / 24.7k |
CooperBench pilot · 20 runs
Paired feature tasks on real Python repos; ten off/on pairs. Within noise. apps/evals/results/pilot-1024/cooper/report.md
| Little Brother off | Little Brother on | |
|---|---|---|
| Merge conflicts | 5 of 10 | 2 of 10 |
| Merged success | 50% | 60% |
| New tokens / run | 25.8k | 26.5k |
| $ / run | $0.18 | $0.18 |
Honest costs: on STALE-bench, agents that were warned solved their own task alone less often (88% → 72% and 94% → 66%), and the hook round trip measured on the larger suite (p95 1.4 s, Node start-up included) misses the 150 ms edit-path target. apps/evals/results/pilot-1024/stale-2/report.md
Haiku agents, four arms
On STALE-bench with Haiku agents, a shared mailbox left 100% of merges broken, the same as no coordination (20 runs each); a hand-written note about the change up front brought it to 45%. After the fixes below, Little Brother caught 46 of 48 planted interface clashes with none mislabelled, but once both agents finish their tasks most merges still break: the caller tends to keep the name its own branch has. Every rerun and its numbers are in the eval report. docs/evals/2026-10-25-report.md
What the runs changed
- Deny once means once. The hook got the open catch back and denied every retry on its stored action; agents told they wouldn't be stopped again were, and gave up.
- Calls inside f-strings. Masking whole Python f-strings hid calls like f"{slugify(name)}", so a signature change never met its caller.
- Say how to finish. Headless agents read a bare denial as "stop the task". Every denial now says it isn't a stop and what to do next.
- Interface clashes have two fixes. The caller writes against the new interface, the changer keeps the old one working; the side found later is held at Stop.
- Hook timeout. A cold start plus a slow backend crossed Claude Code's 3 s limit and dropped the deny; the limit is now 8 s.
5 What's next
- More labelled data from live runs: every decision and every merge outcome becomes a label.
- A fine-tuned cross-encoder for the pairs the tree model leaves to the judge.
- An intervention-value model, so the choice between deny, hint and nothing weighs what each one saves.