# Mixed-piece learning and pawn-oracle experiment

The established pawn model, opening book, tables, and pawn-only decision path
are preserved. This study targets positions containing knights, bishops, or
rooks under `queen-reply-v2`.

## Why an oracle needs caution

The existing pawn engine can supply a cheap feature: remove the extra pieces,
evaluate the resulting pawn structure with its frozen network, and cache the
answer by structure, mover, and en-passant target. The projection's value is
clipped to ±600 before a small coefficient is applied. It never adjudicates
the real game.

Removing pieces changes the game. A mobile knight or bishop supplies reversible
waiting moves, so a pawn-only zugzwang need not survive the projection. Pieces
also block, defend, capture, or race pawns, and can capture the promoted queen.
For a concrete counterexample, White Pa7 against Black Ra1 and Ph2 loses after
a8=Q Rxa8; the pawn-only projection wins for White. Conversely, White Pa2 Nb1
against Black Pa3 wins with Nxa3, while its pawn-only projection is drawn.

The reliable reuse is the existing transition back to the pawn engine when the
**actual** position has no extra pieces. A projected mixed position is a
heuristic, even if its pawn-only answer is solved exactly.

## Budget and data

The run generated **1,200 self-play games** and searched **1,072,377,509 nodes**.
Generation used 7,392.7 summed single-worker seconds (about 2.05 worker-hours),
on the available eight-core CPU environment. No paid GPU service was used.
Actual results were 374 Black wins, 208 rule draws, and 408 White wins; 210 games
remained unresolved at the repetition/ply cap.

After symmetry-aware deduplication there were **27,394 positions** from 1,187
games with eligible samples: 21,992 training positions and 5,402 validation
positions, split by whole games. Of the corpus, 22,411 positions came from
resolved games; 4,983 came from unresolved games. Unresolved examples use a
lower-weight searched target, **never a fabricated draw label**. Most nominal
search depths were three through six, with tactical extensions. These targets
are fallible self-play/search estimates, not perfect-play truth.

The teacher pool includes the original evaluator, hand-crafted features,
projected pawn evaluation, and a tuned expert. Budgets were 8,000 nodes in the
200-game pilot, 16,000 in the next 800 games, and 32,000 in the final 200 games.
The latter used the tuned pilot expert. Sampling covers knights, bishops,
rooks, both minors, all pieces, and random starting subsets. Native training
subsets have material difference at most two points; JavaScript test subsets
also include larger imbalances. Composed imports and every possible custom
configuration are not exhaustively covered.

## Architectures and generalization

Raw inputs are 96 pawn squares + 384 piece squares + 8 en-passant files = 488.
A one-hidden-layer network has `488H + H + H + 1 = 490H + 1` parameters.
The proposed approximately 60K model is therefore precisely **488→128 ReLU→1,
62,721 parameters**. The 8- and 32-unit controls have 3,921 and 15,681 parameters.

The expert has 24 hand-selected features and an optional pawn-oracle feature.
Its 25 coefficients are tuned under bounded material values and a regularizing
prior. The smaller networks use these 25 features plus four total piece counts;
**29→16 ReLU→1 has 497 parameters**. Residual models add at most ±200 score units
to a frozen tuned expert, with a tanh bound. Their 25 base coefficients and the
existing 13,569-parameter pawn network are not newly trained neural parameters.

On the full corpus, selected validation Brier losses against resolved played
outcomes were 0.11217 for the tuned expert, 0.10913 for the 497-parameter network,
0.10979 for its bounded residual version, and 0.13219 for the 62,721-parameter raw
network. The raw network's training loss was 0.07484. Its generalization gap is
substantial; lower validation loss alone also did not guarantee better play.

Final fits took 1.4 seconds for expert coefficients and 5.8 seconds for the
small residual model; the large raw fit took 27.7 seconds. Producing adequate
training targets, rather than fitting the weights, was the bottleneck.

## Discovery tests (not final strength estimates)

All games below swap colors from matched starting positions. W/D/L/U denotes
wins, actual rule draws, losses, and unresolved games for the first competitor.

| Comparison | Search | W / D / L / U |
|---|---|---|
| Pilot 8-unit raw network vs expert + 0.25 pawn projection | Native, 35 ms | 3 / 1 / 42 / 2 |
| Pilot 32-unit raw network vs same expert | Native, 35 ms | 1 / 1 / 45 / 1 |
| Pilot 128-unit raw network vs same expert | Native, 35 ms | 4 / 2 / 42 / 0 |
| Full-corpus 128-unit raw network vs full tuned expert | Native, 40 ms | 6 / 5 / 36 / 1 |
| Initial bounded pawn search at leaves vs cheap pawn projection | Native, 25 ms, 5K-node ceiling | 3 / 1 / 18 / 2 |
| Pawn search with smaller scratch table vs cheap projection | Native, 40 ms | 6 / 5 / 31 / 6 |
| Original evaluator + pawn projection vs original | Native, 40 ms | 13 / 10 / 19 / 6 |
| Original evaluator + projected neural/classical difference vs original | Native, 40 ms | 10 / 13 / 18 / 7 |
| Pawn-search root ordering hint vs original | Native, 40 ms | 20 / 10 / 27 / 15 |
| Tuned pilot expert, oracle on vs off | JavaScript, 120 ms | 21 / 21 / 18 / 4 |
| Intermediate 497-parameter residual vs intermediate tuned expert | JavaScript, 120 ms | 22 / 8 / 13 / 5 |

The oracle-on/off score among resolved games was 52.5%, with a paired 95%
bootstrap interval of 42.9–62.3%. Accounting for unresolved games without
assuming draws bounds its overall score at 49.2–55.5%. That is **not evidence
of a reliable independent oracle gain**. Costlier pawn searches performed
worse at the tested budgets. The raw networks lost even while searching more
nominal plies on average than their expert opponents.

The intermediate residual model's apparent gain was a selection result, with
a wide interval. Fresh confirmation games determine the deployment decision.

## Confirmation and deployment

The final comparison freezes model files and uses fresh randomized starting
positions with colors swapped, equal per-move clocks, the actual JavaScript
search, exact pawn tables, and the app's recent-history preference. Oracle
caches are cleared between moves to match the app's worker lifetime. Separate
800 ms checks use the normal starting presets. Raw game traces and a config
with checkpoint hashes are saved beside this report.

The selected checkpoint is `mixed-final-res16`: 497 neural parameters, a frozen
25-coefficient expert base, and the existing frozen pawn network. Its correction
is bounded to ±200. The full-data and intermediate residual models tied 8 wins,
5 draws, 8 losses, and 3 unresolved games in their direct selection match; the
full-data checkpoint was retained for its broader training coverage. The final
expert opponent is the independently tuned `mixed-interim-linear` checkpoint.

The following **324 fresh confirmation games** were held out of selection.
Randomized-position tests use 120 ms per move; the separate standard-preset
checks use 800 ms. All clocks are equal within a match.

| Comparison | Games | W / D / L / U | Score on resolved games | Paired 95% interval, resolved only |
|---|---:|---|---:|---|
| Hybrid vs optimized expert | 120 | 46 / 28 / 29 / 17 | 58.3% | 51.0–65.8% |
| Hybrid vs previous mixed-piece evaluator | 120 | 74 / 16 / 18 / 12 | 75.9% | 69.4–82.2% |
| Optimized expert vs previous evaluator | 60 | 24 / 9 / 19 / 8 | 54.8% | 45.2–64.7% |
| Hybrid vs optimized expert (800 ms) | 12 | 5 / 3 / 2 / 2 | 65.0% | 50.0–81.8% |
| Hybrid vs previous evaluator (800 ms) | 12 | 8 / 3 / 1 / 0 | 79.2% | 62.5–95.8% |

The hybrid's overall score, without assuming any outcome for unfinished games,
is bounded at **50.0–64.2% against the optimized expert** and **68.3–78.3% against
the previous evaluator** at 120 ms. The corresponding resolved-game intervals
are conditional on a game finishing. They must not be read as confidence
intervals for the unresolved games' unknown outcomes. The edge over the expert
is modest; the improvement over the old mixed-piece evaluator is much clearer.
The 800 ms sets are supporting checks, not large independent rating studies.

The hybrid's 120 ms results against the expert by preset were:

| Preset | W / D / L / U |
|---|---|
| Knights | 9 / 2 / 7 / 2 |
| Bishops | 8 / 1 / 6 / 5 |
| Rooks | 6 / 9 / 0 / 5 |
| Both minor pieces | 7 / 3 / 8 / 2 |
| All pieces | 7 / 7 / 4 / 2 |
| Random subsets | 9 / 6 / 4 / 1 |

This does not establish an improvement in every preset: the both-minors slice,
for example, was slightly negative at the short clock. No human/NM rating is
claimed. The new default applies only while extra pieces remain; the established
pawn weights, opening book, tablebase assets, and fixed-node pawn decisions
are unchanged. Rules, imports/exports, and site audience are preserved.


## Verification and limits

The native port agreed with the JavaScript rules on 5,000 positions and 77,201
legal continuations. Model/feature differential checks each covered 1,500
positions and agreed within at most two score units (usually exactly). The
first 1,200 native benchmark games, comprising 64,107 plies, also replayed to
the same outcomes through the JavaScript rules.

All **75 automated tests pass**. Regression tests cover projected-oracle counterexamples, reflection symmetry,
cache keys, pure-pawn delegation, immutable pawn asset hashes, and twelve
unchanged fixed-node pawn decisions. The last-pawn promotion and queen-capture
rules remain covered by the existing suite. Interactive browser/screenshot QA
was unavailable in this environment; the engine tests run in Node.

These comparisons establish neither perfect play nor a human rating. Repeated
and capped games remain unresolved, not automatically drawn. Multiple
discovery experiments inform selection, and the final matched-opening results
must be distinguished from those exploratory results. A larger mixed-piece
model would need better and more diverse targets, or stronger geometric
structure, before its extra capacity would be justified by this experiment.

Programs, configurations, all candidate checkpoints, compressed input data,
prepared datasets, raw traces, and validation logs are versioned in the source
repository. See `training/mixed/README.md` for exact encodings and commands.