Neural Recombination Runbook

Step-by-step guide for repeating the full distillation → quantization → crossover → fine-tune → validation pipeline from AgentFarm#8.

Table of Contents

1. Prerequisites
2. Quick Reference: Pipeline Order
3. Stage 1 — Distillation
4. Stage 2 — Post-Training Quantization (PTQ)
5. Stage 3 — Quantization-Aware Training (QAT) — Optional
6. Stage 4 — Crossover + Fine-tuning
7. Stage 5 — Validation
8. Optional: Compare Distillation Modes
9. Parameter Reference
10. Tuning Guide
11. Copy-Paste Recipes
12. Generalization: Holdout & Domain-Shift Evaluation
13. Publication Ablations
14. Qualitative Error Analysis for Recombined Networks
15. Multi-Generation Crossover Search

1. Prerequisites

Environment

python -m venv venv
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

Python 3.8+ required; 3.9+ recommended. All scripts below assume the repo root is your working directory and that the venv is active.

Parent checkpoints

Each pipeline run requires two pre-trained parent BaseQNetwork state-dicts — parent_A.pt and parent_B.pt. These are the teachers for distillation. Architecture dimensions (input_dim, output_dim, hidden_size) must stay consistent across every stage.

The default architecture shipped with farm/config/default.yaml is:

If your parents use different dimensions, pass the corresponding flags to every script.

State buffer

All scripts share a common evaluation dataset: an (N, input_dim) float32 NumPy array.

• Synthetic (quickest): generated at runtime via --n-states + --seed; always reproducible but not representative of real agent behaviour.
• Real replay buffer: pass --states-file path/to/states.npy. The .npy file must have shape (N, input_dim) in float32. Using the same distribution as training/deployment gives the most realistic validation metrics.

Reproducibility rule: use the same --states-file (or the same --n-states/--seed pair) across all stages so that metrics are comparable.

2. Quick Reference: Pipeline Order

parent_A.pt ─┐
             ├─► run_distillation.py ─► student_A.pt ─┐
parent_B.pt ─┘                          student_B.pt ─┘
                                               │
                              (PTQ) quantize_distilled.py ─► student_A_int8.pt
                         (QAT) qat_distilled.py    ─► student_A_qat_int8.pt
                                               │
                         finetune_child.py (crossover inside) ─► child_finetuned.pt
                                               │
                      validate_distillation.py · validate_quantized.py
                      validate_recombination.py

Each stage writes checkpoints and companion JSON metadata (*.pt.json) that become inputs for the next stage.

3. Stage 1 — Distillation

Script: scripts/run_distillation.py

Goal: train a smaller StudentQNetwork to reproduce the Q-value distribution of each frozen parent (BaseQNetwork).

Minimal synthetic run

python scripts/run_distillation.py \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --n-states 2000 \
  --seed 42 \
  --output-dir checkpoints/distillation

With a real replay buffer

python scripts/run_distillation.py \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --states-file data/replay_states.npy \
  --seed 42 \
  --output-dir checkpoints/distillation

Expected shape of replay_states.npy: (N, 8) float32 (or whatever your input_dim is).

Key parameters

Outputs

checkpoints/distillation/
  student_A.pt          # StudentQNetwork state dict
  student_A.pt.json     # config + per-epoch metrics
  student_B.pt
  student_B.pt.json

4. Stage 2 — Post-Training Quantization (PTQ)

Script: scripts/quantize_distilled.py

Goal: compress student_*.pt to int8 with no re-training. Start here before trying QAT.

Minimal dynamic PTQ

python scripts/quantize_distilled.py \
  --checkpoint-dir checkpoints/distillation \
  --input-dim 8 --output-dim 4 --parent-hidden 64 \
  --mode dynamic \
  --output-dir checkpoints/quantized

Static PTQ (with calibration data)

python scripts/quantize_distilled.py \
  --checkpoint-dir checkpoints/distillation \
  --states-file data/replay_states.npy \
  --mode static \
  --calibration-batches 10 \
  --calibration-batch-size 64 \
  --output-dir checkpoints/quantized

Key parameters

Outputs

checkpoints/quantized/
  student_A_int8.pt        # Quantized model (CPU int8 pickle)
  student_A_int8.pt.json   # QuantizationConfig + timing
  student_B_int8.pt
  student_B_int8.pt.json

5. Stage 3 — Quantization-Aware Training (QAT) — Optional

Script: scripts/qat_distilled.py

When to use QAT instead of PTQ:

Use QAT when PTQ action agreement (reported by validate_quantized.py) falls below your target threshold (e.g. < 90%). QAT adds a short training pass with fake quantization, recovering accuracy at the cost of a few extra minutes.

Minimal QAT run

python scripts/qat_distilled.py \
  --checkpoint-dir checkpoints/distillation \
  --input-dim 8 --output-dim 4 --parent-hidden 64 \
  --epochs 5 \
  --n-states 2000 \
  --seed 42 \
  --output-dir checkpoints/qat

Omit --no-convert (default) to also produce the converted int8 checkpoint.

Key parameters

Outputs

checkpoints/qat/
  student_A_qat.pt           # Float QAT checkpoint
  student_A_qat.pt.json
  student_A_qat_int8.pt      # Converted int8 (same format as PTQ output)
  student_A_qat_int8.pt.json

6. Stage 4 — Crossover + Fine-tuning

Script: scripts/finetune_child.py

Goal: blend two parent state dicts into a child via a crossover strategy, then fine-tune the child against a frozen reference (parent A) using a distillation-style loss.

Minimal synthetic run

python scripts/finetune_child.py \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --crossover-mode weighted \
  --crossover-alpha 0.5 \
  --crossover-seed 42 \
  --n-states 2000 \
  --seed 42 \
  --output-dir checkpoints/finetune

With a replay buffer and YAML overrides

python scripts/finetune_child.py \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --crossover-mode random \
  --crossover-alpha 0.5 \
  --crossover-seed 0 \
  --states-file data/replay_states.npy \
  --config-yaml farm/config/default.yaml \
  --epochs 10 \
  --lr 5e-4 \
  --output-dir checkpoints/finetune

YAML defaults (farm/config/default.yaml)

The crossover_child_finetune section provides all fine-tuning defaults:

crossover_child_finetune:
  learning_rate: 0.001
  epochs: 5
  batch_size: 32
  max_grad_norm: 1.0
  val_fraction: 0.1
  seed: null
  loss_fn: kl
  temperature: 3.0
  temp_decay: 1.0          # per-epoch temperature multiplier; 1.0 = no decay
  alpha: 1.0               # soft/hard blend (1.0 = pure KL, 0.0 = pure CE)
  lr_schedule_patience: 0  # ReduceLROnPlateau patience; 0 = disabled
  lr_schedule_factor: 0.5  # LR reduction factor when plateau detected
  quantization_applied: none   # none | ptq_dynamic | ptq_static | qat_float
  optimizer: adam
  optimizer_kwargs: {}
  early_stopping_patience: 0   # 0 = disabled

CLI flags such as --lr, --epochs, --alpha override the YAML values when specified.

Key parameters

Crossover modes

Outputs

checkpoints/finetune/
  child.pt                  # Raw crossover child (pre fine-tune)
  child.pt.json
  child_finetuned.pt        # Fine-tuned child
  child_finetuned.pt.json

7. Stage 5 — Validation

7.1 Distillation quality

Script: scripts/validate_distillation.py

Checks KL divergence, MSE, MAE, cosine similarity, top-k agreement, latency ratio, and robustness slices between parent and student.

python scripts/validate_distillation.py \
  --checkpoint-dir checkpoints/distillation \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --n-states 2000 --seed 42 \
  --report-dir reports/distillation

Key threshold flags (all have sensible defaults; override to tighten):

7.2 Quantization fidelity

Script: scripts/validate_quantized.py

Compares float student vs int8 student on agreement, Q-error, latency, and memory.

python scripts/validate_quantized.py \
  --float-dir checkpoints/distillation \
  --quant-dir checkpoints/quantized \
  --n-states 2000 --seed 42 \
  --report-dir reports/quantized

Key threshold flags:

7.3 Recombination quality

Script: scripts/validate_recombination.py

Evaluates the fine-tuned child against both parents.

python scripts/validate_recombination.py \
  --checkpoint-dir checkpoints/finetune \
  --parent-a-ckpt checkpoints/parent_A.pt \
  --parent-b-ckpt checkpoints/parent_B.pt \
  --child-ckpt checkpoints/finetune/child_finetuned.pt \
  --n-states 2000 --seed 42 \
  --include-parent-baseline \
  --report-dir reports/recombination

For quantized children add --child-quantized (and --parent-a-quantized / --parent-b-quantized if parents are also int8).

Key threshold flags:

“Good enough” heuristic: aim for child-vs-A and child-vs-B top-1 agreement both ≥ 0.7, with neither collapsing to one parent. The primary_metric = min(agreement_A, agreement_B) used by run_crossover_search.py captures this directly.

Case-level analysis: for per-state disagreements, logit summaries, worst-k states, and hidden-layer activations see § 14 — Qualitative Error Analysis and scripts/analyze_recombination.py.

Validation report layout

Each script writes a JSON report alongside a human-readable markdown summary. JSON keys of interest:

• action_agreement — top-1 agreement between models
• oracle_action_agreement — agreement when both models are uncertain
• kl_divergence, mse, mae, cosine_similarity — Q-value closeness metrics
• latency_ms_median / latency_ratio — speed comparison

8. Optional: Compare Distillation Modes

Script: scripts/compare_distillation_modes.py

Runs hard-only (alpha=0), soft-only (alpha=1), and blended distillation back-to-back with a shared frozen teacher and state buffer so results are directly comparable.

python scripts/compare_distillation_modes.py \
  --seed 42 --epochs 10 --n-states 2000 \
  --json-out reports/distillation_mode_comparison.json

See docs/distillation_soft_label_comparison.md for recorded results and discussion.

9. Parameter Reference

Architecture consistency

The following dimensions must match across all stages:

Seed inventory

crossover_child_finetune YAML keys

quantization_applied values: none (default, float path), ptq_dynamic, ptq_static, qat_float — when not none, FineTuner replaces Linear layers with WeightOnlyFakeQuantLinear; call convert() + save_quantized() after finetune() for int8 output.

10. Tuning Guide

State data choice

• Use a real replay buffer whenever possible. Synthetic standard-normal states cover the full input range but may not reflect the distribution your agents encounter at inference time. Validation metrics on synthetic states can be optimistic.
• The .npy file should be float32, shape (N, input_dim). Prefer N ≥ 2 000 for stable statistics; 10 000+ for final validation.
• Keep the same states file across all stages (or at least the same --seed) so that reported metrics are on a common distribution.

Distillation

• Temperature (--temperature, default 3.0): higher values flatten the teacher’s distribution, making inter-action confidence ordering more visible. Range 2.0–6.0 is typical; reduce toward 1.0 if the student converges too slowly.
• Alpha (--alpha, default 1.0): 1.0 = pure KL soft loss; 0.0 = pure cross-entropy on the argmax. Start with 1.0; blend toward 0.5 if action agreement is high but hard-label accuracy is poor.
• Learning rate instability: if training loss oscillates or diverges, lower --lr (try 5e-4) and ensure --max-grad-norm 1.0 is in effect.
• Low agreement after training: increase --epochs (try 20), or provide a larger / more representative state buffer.
• --loss-fn mse: simpler objective; useful for debugging, but KL is generally better for Q-value distributions.

Post-Training Quantization

• Try dynamic PTQ first — zero training cost, typically ≥ 90 % action agreement. Only invest in static PTQ or QAT if validate_quantized.py reports agreement below your target.
• Static PTQ: requires calibration data that reflects real input distribution; use --states-file. Use --calibration-batches 10–50 and --calibration-batch-size 64–256.
• Backend choice: on Intel CPUs fbgemm is often fastest; on ARM/mobile use qnnpack; auto selects automatically. Benchmark with --throughput-batch-size in validate_quantized.py.
• QAT vs PTQ decision rule: if PTQ action_agreement < 0.90, try QAT with --epochs 5 --learning-rate 1e-4. Use --loss-fn mse for QAT (default) unless the teacher distribution is very soft.

Crossover

• weighted mode is the smoothest starting point: --crossover-alpha 0.5 gives a midpoint blend. Move alpha toward 0.3 or 0.7 to bias toward one parent.
• random mode produces the most diverse children but highest variance across seeds. Run three seeds (--crossover-seed 0,1,2 via separate invocations) to gauge variance before committing to one.
• layer mode preserves structural coherence (each Linear + LayerNorm pair from the same parent) at the cost of diversity — only two possible children per pair. Useful when the network is sensitive to feature-scaling mismatches.
• If child collapses to one parent: primary_metric = min(agreement_A, agreement_B) will be low. Try weighted at 0.5, or random with a different seed.

Fine-tuning

• Reference teacher: finetune_child.py always uses parent A as the fine-tune teacher. This biases the child toward A’s behaviour; if you want a more balanced child, check parent B agreement explicitly with validate_recombination.py --include-parent-baseline.
• Loss function: kl with temperature 3.0 and alpha 1.0 mirrors the distillation loss and tends to produce soft, calibrated Q-values. Use mse for a simpler target.
• LR schedule: enable --lr-patience 3 --lr-factor 0.5 if validation loss plateaus early. This is disabled by default to keep runs short.
• Early stopping: --early-stopping-patience 5 prevents overfitting on small state buffers; safe to enable for production runs.
• quantization_applied: set to ptq_dynamic, ptq_static, or qat_float only when you intend to produce an int8 fine-tuned child. Requires calling FineTuner.convert() + save_quantized() programmatically after the script; for a pure float32 child leave as none.

Validation

• Dimension/architecture mismatches are the most common failure mode. Always pass the same --input-dim, --output-dim, and --parent-hidden (or --hidden-size) that were used at distillation time. The checkpoint JSON metadata (*.pt.json) records these for reference.
• Agreement vs oracle agreement: action_agreement counts top-1 matches; oracle_action_agreement counts matches only when both models are confident. Low oracle agreement with high action agreement usually indicates one model is uncertain overall.
• Threshold calibration: default thresholds in the validation scripts are conservative. After a successful baseline run you can tighten --min-action-agreement toward 0.85–0.95 for production gates.
• Using the same state buffer as training: evaluation metrics will be optimistically inflated if the same states are used for both training and validation. Prefer a held-out test split by passing a separate .npy file to the validation scripts.

11. Copy-Paste Recipes

Recipe A — Minimal synthetic run (no parent checkpoints required for architecture test)

# 1. Distil (synthetic states, no real parents — useful for smoke test)
python scripts/run_distillation.py \
  --n-states 2000 --seed 42 \
  --epochs 5 \
  --output-dir checkpoints/distillation

# 2. PTQ
python scripts/quantize_distilled.py \
  --checkpoint-dir checkpoints/distillation \
  --n-states 2000 --seed 42 \
  --mode dynamic \
  --output-dir checkpoints/quantized

# 3. Validate distillation
python scripts/validate_distillation.py \
  --checkpoint-dir checkpoints/distillation \
  --n-states 2000 --seed 42 \
  --report-dir reports/distillation

# 4. Validate quantization
python scripts/validate_quantized.py \
  --float-dir checkpoints/distillation \
  --quant-dir checkpoints/quantized \
  --n-states 2000 --seed 42 \
  --report-dir reports/quantized

# 5. Crossover + fine-tune
python scripts/finetune_child.py \
  --parent-a-ckpt checkpoints/distillation/student_A.pt \
  --parent-b-ckpt checkpoints/distillation/student_B.pt \
  --crossover-mode weighted --crossover-alpha 0.5 \
  --n-states 2000 --seed 42 \
  --output-dir checkpoints/finetune

# 6. Validate recombination
python scripts/validate_recombination.py \
  --parent-a-ckpt checkpoints/distillation/student_A.pt \
  --parent-b-ckpt checkpoints/distillation/student_B.pt \
  --child-ckpt checkpoints/finetune/child_finetuned.pt \
  --n-states 2000 --seed 42 \
  --report-dir reports/recombination

Recipe B — Full run with real parent checkpoints and replay buffer

PARENTS=checkpoints/parents
STATES=data/replay_states.npy   # shape (N, 8) float32
OUT=checkpoints/run1

# 1. Distil
python scripts/run_distillation.py \
  --parent-a-ckpt $PARENTS/parent_A.pt \
  --parent-b-ckpt $PARENTS/parent_B.pt \
  --states-file $STATES --seed 42 \
  --temperature 3.0 --alpha 1.0 \
  --epochs 20 --lr 1e-3 \
  --output-dir $OUT/distillation

# 2. Validate distillation
python scripts/validate_distillation.py \
  --checkpoint-dir $OUT/distillation \
  --parent-a-ckpt $PARENTS/parent_A.pt \
  --parent-b-ckpt $PARENTS/parent_B.pt \
  --states-file $STATES --seed 42 \
  --report-dir $OUT/reports/distillation

# 3. PTQ (try dynamic first)
python scripts/quantize_distilled.py \
  --checkpoint-dir $OUT/distillation \
  --states-file $STATES --seed 42 \
  --mode dynamic \
  --output-dir $OUT/quantized

# 4. Validate quantization
python scripts/validate_quantized.py \
  --float-dir $OUT/distillation \
  --quant-dir $OUT/quantized \
  --states-file $STATES --seed 42 \
  --report-dir $OUT/reports/quantized

# (Optional) If PTQ agreement < 0.90, run QAT instead:
# python scripts/qat_distilled.py \
#   --checkpoint-dir $OUT/distillation \
#   --states-file $STATES --seed 42 \
#   --epochs 5 --learning-rate 1e-4 \
#   --output-dir $OUT/qat

# 5. Crossover + fine-tune (using float parents)
python scripts/finetune_child.py \
  --parent-a-ckpt $PARENTS/parent_A.pt \
  --parent-b-ckpt $PARENTS/parent_B.pt \
  --crossover-mode weighted --crossover-alpha 0.5 \
  --crossover-seed 42 \
  --states-file $STATES --seed 42 \
  --epochs 10 --lr 5e-4 \
  --lr-patience 3 --early-stopping-patience 5 \
  --output-dir $OUT/finetune

# 6. Validate recombination
python scripts/validate_recombination.py \
  --parent-a-ckpt $PARENTS/parent_A.pt \
  --parent-b-ckpt $PARENTS/parent_B.pt \
  --child-ckpt $OUT/finetune/child_finetuned.pt \
  --states-file $STATES --seed 42 \
  --include-parent-baseline \
  --report-dir $OUT/reports/recombination

12. Generalization: Holdout & Domain-Shift Evaluation

Script: scripts/eval_generalization.py

Standard validation metrics (Sections 7.1–7.3) are measured on a single state buffer that may overlap with calibration or training data. For publication-grade generalization claims, you need:

1. A held-out test split that was never used for training or calibration.
2. An optional domain-shift evaluation generated by perturbing the holdout states (for example, sensor noise or input scaling).

eval_generalization.py automates both steps by:

• Splitting the state buffer into an in-distribution (ID) and holdout subset.
• Optionally perturbing the holdout set with Gaussian noise or input scaling.
• Running RecombinationEvaluator on each subset and writing a per-set JSON report plus a combined generalization_summary.json.

At present, the CLI documents perturbation-based domain shift only; it does not expose a flag for loading a separate shifted .npy states file.

Library helpers

The split and perturbation logic is available as standalone functions in farm.core.decision.training.holdout_utils:

All helpers are also re-exported from farm.core.decision.training.

Minimal synthetic run (synthetic states, existing checkpoints required)

This example uses synthetic evaluation states, but it still requires trained parent A, parent B, and child checkpoints from the Recipe A workflow in Section 11. Replace paths as needed.

python scripts/eval_generalization.py \
  --parent-a-ckpt checkpoints/distillation/student_A.pt \
  --parent-b-ckpt checkpoints/distillation/student_B.pt \
  --child-ckpt    checkpoints/finetune/child_finetuned.pt \
  --n-states 2000 --seed 42 \
  --holdout-fraction 0.2 \
  --report-dir reports/generalization

Output:

reports/generalization/
  id_report.json           # in-distribution split report
  holdout_report.json      # held-out split report
  generalization_summary.json

With a real replay buffer

python scripts/eval_generalization.py \
  --parent-a-ckpt checkpoints/parents/parent_A.pt \
  --parent-b-ckpt checkpoints/parents/parent_B.pt \
  --child-ckpt    checkpoints/finetune/child_finetuned.pt \
  --states-file   data/replay_states.npy \
  --holdout-fraction 0.2 \
  --report-dir    reports/generalization

With Gaussian-noise domain shift

python scripts/eval_generalization.py \
  --parent-a-ckpt checkpoints/parents/parent_A.pt \
  --parent-b-ckpt checkpoints/parents/parent_B.pt \
  --child-ckpt    checkpoints/finetune/child_finetuned.pt \
  --states-file   data/replay_states.npy \
  --holdout-fraction 0.2 \
  --shift-type    gaussian_noise \
  --shift-std     0.1 \
  --shift-seed    0 \
  --report-dir    reports/generalization

Output adds reports/generalization/shifted_report.json.

With input-scaling domain shift

python scripts/eval_generalization.py \
  --parent-a-ckpt checkpoints/parents/parent_A.pt \
  --parent-b-ckpt checkpoints/parents/parent_B.pt \
  --child-ckpt    checkpoints/finetune/child_finetuned.pt \
  --states-file   data/replay_states.npy \
  --shift-type    input_scaling \
  --shift-scale-factor 2.0 \
  --report-dir    reports/generalization

Key flags

Reading the generalization_summary.json

{
  "overall_passed": true,
  "report_only": false,
  "holdout_fraction": 0.2,
  "shift_type": "gaussian_noise",
  "sets": {
    "in_distribution": {
      "child_agrees_with_parent_a": 0.82,
      "child_agrees_with_parent_b": 0.79,
      "oracle_agreement": 0.91,
      "n_states": 1600,
      "passed": true
    },
    "holdout": {
      "child_agrees_with_parent_a": 0.80,
      "child_agrees_with_parent_b": 0.77,
      "oracle_agreement": 0.89,
      "n_states": 400,
      "passed": true
    },
    "shifted": {
      "child_agrees_with_parent_a": 0.73,
      "child_agrees_with_parent_b": 0.70,
      "oracle_agreement": 0.84,
      "n_states": 400,
      "passed": true,
      "shift_type": "gaussian_noise"
    }
  }
}

A meaningful generalization drop is when holdout or shifted agreement falls more than ~5 pp below the ID score. If this happens, consider:

• Training on a larger or more diverse replay buffer.
• Increasing the holdout fraction to detect over-fitting earlier in development.
• Tuning the crossover alpha or fine-tuning LR to reduce ID–holdout gap.

13. Publication Ablations

Script: scripts/run_recombination_ablation.py

For reproducible paper tables and CI-style regression of the full pipeline, use the unified ablation runner. A single invocation sweeps multiple conditions (e.g. distill-only, distill+quantize, or the full pipeline) across a list of seeds and writes every result into a structured results/ tree together with a consolidated CSV and Markdown summary table. If you provide a shared states_file, that same state buffer is reused across seeds and conditions. If states_file is omitted, the runner generates synthetic states per seed, so cross-seed results are not directly comparable unless you supply a common state buffer.

Quick start (no config file needed)

# Dry-run: validate plan, write stub summary, no training
python scripts/run_recombination_ablation.py --smoke-test --dry-run

# Smoke-test: tiny synthetic run (2 seeds × 3 conditions, 50 states, 2 epochs)
python scripts/run_recombination_ablation.py --smoke-test --results-dir /tmp/ablation_smoke

Full run from a config file

python scripts/run_recombination_ablation.py --config ablation.yaml

The config file is YAML (recommended) or JSON. A minimal example:

seeds: [0, 1, 2]
n_states: 2000
states_file: ""           # leave empty to synthesise per-seed (supply a .npy path for comparable cross-seed results)
input_dim: 8
output_dim: 4
hidden_size: 64
results_dir: results/ablation

conditions:
  - name: distill_only
    stages: [distill]
  - name: distill_quantize
    stages: [distill, quantize]
  - name: full_pipeline
    stages: [distill, quantize, crossover, compare]

distillation:
  epochs: 20
  temperature: 3.0
  alpha: 1.0
  lr: 0.001
  batch_size: 32

quantization:
  mode: dynamic

crossover:
  mode: weighted
  alpha: 0.5

comparison:
  report_only: true

Output layout

results/ablation/
  distill_only/
    seed_0/student_A.pt  student_B.pt
    seed_1/...
    seed_2/...
  distill_quantize/
    seed_0/student_A.pt  student_B.pt  student_A_int8.pt  student_B_int8.pt
    ...
  full_pipeline/
    seed_0/student_A.pt  student_B.pt  student_A_int8.pt  student_B_int8.pt
             child_finetuned.pt  compare_child_vs_students.json
    ...
  ablation_summary.csv          ← consolidated table (paste into spreadsheet)
  ablation_summary.md           ← Markdown version (paste into GitHub issues)

Per-condition stage overrides

Each condition can override any global distillation / quantization / crossover / comparison setting:

conditions:
  - name: high_temp_distill
    stages: [distill, crossover, compare]
    distillation:
      temperature: 6.0   # overrides global temperature: 3.0
      epochs: 30

Valid stages

Stages are always applied in the order listed above regardless of declaration order in the config. Parse-time rules: quantize and crossover require distill; compare requires crossover (there must be a child to score).

Dry-run mode

python scripts/run_recombination_ablation.py --config ablation.yaml --dry-run

Prints the full execution plan (conditions × seeds × stages × directories) and writes a stub ablation_summary.md / ablation_summary.csv without running any training. Use this to verify the config before a long run.

Using a shared real replay buffer

Set states_file in the config to a .npy file of shape (N, input_dim) float32. All seeds and conditions will use the same state file, ensuring metrics are comparable across the ablation.

states_file: data/replay_states.npy

Reading the summary table

The Markdown summary table (ablation_summary.md) contains one row per (condition, seed) pair. Key columns:

child_vs_ref_*_agreement columns are populated only when the compare stage is included. Conditions without a compare stage show n/a.

14. Qualitative Error Analysis for Recombined Networks

Script: scripts/analyze_recombination.py
Python API: farm.core.decision.training.recombination_analysis

The aggregate fidelity report from validate_recombination.py (§ 7.3) shows mean agreement across all states. For publication or debugging you often need case-level insight: which states does the child get wrong, and is the disagreement systematic? analyze_recombination.py provides this.

What it produces

Minimal run

python scripts/analyze_recombination.py \
  --checkpoint-dir checkpoints/finetune \
  --parent-a-ckpt  checkpoints/parent_A.pt \
  --parent-b-ckpt  checkpoints/parent_B.pt \
  --child-ckpt     checkpoints/finetune/child_finetuned.pt \
  --states-file    data/replay_states.npy \
  --output-dir     reports/analysis

With logits, worst-10 states, and activation export

python scripts/analyze_recombination.py \
  --checkpoint-dir      checkpoints/finetune \
  --states-file         data/replay_states.npy \
  --include-logits \
  --worst-k             10 \
  --worst-k-criterion   max_kl \
  --activations-out     reports/analysis/child_activations.npy \
  --activation-layer-index 4 \
  --activation-max-states  500 \
  --output-dir          reports/analysis

--activation-layer-index selects a sub-module by its index in list(model.modules()). For BaseQNetwork:

Python API

import numpy as np
from farm.core.decision.training.recombination_analysis import (
    extract_disagreements,
    worst_k_states,
    export_disagreements_csv,
    export_disagreements_json,
    extract_activations,
)

states = np.load("data/replay_states.npy")

records = extract_disagreements(
    parent_a, parent_b, child, states,
    include_logits=True,
    k_values=[1, 2, 3],
)

# Worst-10 states by maximum KL divergence across the two parents
worst = worst_k_states(records, k=10, criterion="max_kl")

export_disagreements_csv(records, "reports/analysis/disagreements.csv")
export_disagreements_json(records, "reports/analysis/disagreements.json")

# Memory-bounded activation export (first hidden ReLU, max 500 states)
acts = extract_activations(child, states, layer_index=4, max_states=500)
np.save("reports/analysis/child_activations.npy", acts)

Worst-k criteria

KL columns: kl_child_vs_parent_a / _b are KL(parent ‖ child) over action softmaxes (parent as the reference distribution). The field names are historical; compare to other tools that report KL(child ‖ parent) carefully.

Integration with validate_recombination.py

Run validate_recombination.py first to check that aggregate fidelity meets thresholds, then run analyze_recombination.py to drill into problem states. Both share the same architecture flags and checkpoint conventions.

15. Multi-Generation Crossover Search

Script: scripts/run_multi_gen_search.py
Python API: farm.core.decision.training.crossover_search.run_multi_generation_search, GenerationConfig

This mode runs :func:~farm.core.decision.training.crossover_search.run_crossover_search repeatedly. After each generation, the best child checkpoint becomes parent A for the next generation; parent B is chosen from the leaderboard according to selection_strategy (see below). Optional Gaussian mutation perturbs promoted parents’ weights before the next generation’s crossovers.

Semantics (parent selection)

Per-generation RNG (GenerationConfig.seed)

In the CLI, --seed is forwarded to GenerationConfig.seed and seeds synthetic parent/state generation when you omit checkpoint/state files. When seed is set, generation g adds seed + g to every non-None crossover recipe seed and fine-tune regime seed in the shared SearchConfig. When seed is None, recipe and regime seeds are used exactly as configured. Mutation RNG seeds combine MutationConfig.seed, GenerationConfig.seed, and the generation index; see the GenerationConfig docstring in code.

Minimal CLI example

python scripts/run_multi_gen_search.py \
  --search-space minimal \
  --max-runs 3 \
  --num-generations 3 \
  --seed 1000 \
  --run-dir runs/multi_gen_smoke

For a single generation with the full run_crossover_search.py flag surface (eval batch size, workers, recombination thresholds, etc.), use scripts/run_crossover_search.py. run_multi_gen_search.py uses a smaller argument set focused on multi-gen knobs; extend the Python API if you need full parity.

Related Documentation