32 Chapter 32 — The Laboratory: Reproducing the Book's Claims
One runner, every lab, and the status table that separates measured from pending
32.1 The Book Is Runnable
A reference book about automation must itself be automated. This chapter is the single entry point: lab/run_everything.py executes every lab in the book and writes a status table — which pass, which need Colab, which skip. The runner is the reproducibility contract: a claim in any chapter resolves to a lab; the lab resolves to a pass/fail row in this chapter's report.
The design follows the lab-per-chapter production model proven by fast.ai's fastbook and nbdev (source-backed): chapters there are executable notebooks, CI runs the notebook assertions, and a broken lab is by definition a false claim. Our stack is the plain-Python equivalent of that idea. Every chapter's lab box names a script under lab/ or experiments/; every script writes a metrics.json artifact; every numeric claim in the prose is bound to one of those artifacts through the claims register (32.1b). The evaluation-driven-development rule from the research program applies verbatim: the spec — the claim row — exists before the prose that quotes it, and any paragraph whose claim row is empty fails review.
The whole laboratory runs against two fixtures, one per sport, and the choice of fixtures is itself part of the method. The pickleball fixture is pb-003: a 900-frame, 25 fps, 1920x1080 derived clip from a PPA broadcast (Waters/Bright v Black/Todd), with six golden frames, a v1 label schema (12 court keypoints, 4 players, ball state, 12 event types), and a licensing caveat — labels are CC-BY-4.0, frames are not redistributable, so the dataset ships labels plus frame indices plus the derived clip's SHA-256 and readers point at their own licensed copy. The rugby-league fixture is nrl-001: 750 frames at 25 fps of open play from the Storm v Panthers 2024 grand-final highlights — broadcast footage with a panning, zooming camera, 13.2 persons per frame on average (E08, measured), ruck-phase occlusion bursts, and none of the fixed-camera comfort pb-003 offers. The pairing is deliberate: pb-003 is the controlled lane where methods can be measured to the decimal; nrl-001 is the adversarial lane where the same methods meet broadcast reality.
Everything here is measured — the runner's own output (the pb-003 fixture's 19/20 fast labs pass; the 20th was fixed in this session; the rugby-lane labs C07/C13 are the first-pass evidence), or explicitly pending (the Colab lanes). Nothing in this chapter is aspirational.
32.1b The Claims Register: E-ID Mapping and the Audit
The claims register (lab/claims_register.py, gate G0.1) is the book's evidence spine. It is a canonical table of 24 claim rows — C-01 through C-24 — each a tuple of (claim id, chapter, asserted claim, experiment directory, artifact path, value key, status). The experiment directory is the E-ID link: claim C-15 ("the 3-frame temporal energy filter lifts ball-detection precision from 4.2% to 81.4%") resolves to experiments/e10-ball-temporal/outputs/metrics.json at the dot-path key temporal_3frame_difference.true_ball_precision_at_50, expected value 0.814. The auditor loads the artifact, walks the key path, and compares against the register's expected value with a tolerance of 1e-6. That mechanic — claim, artifact, key, expected value — is what turns a sentence in a chapter into a checkable fact.
The register recognizes three honest dispositions, and the distinctions matter more than the pass count. VERIFIED means the artifact exists and the value matches: C-01 (generic YOLO11n on pb-003 yields 69 raw person boxes, 11 after court filtering — 84% noise rejection, E01), C-05 (YOLO11-pose at 82 fps on pb-003, E03), C-14 (BoT-SORT ReID cuts pickleball ID switches 14 → 2 and lifts HOTA to 81.4%, E09). BLOCKED means the claim is known false and the artifact is the disproof: C-04 ("ByteTrack maintains identity across the 30-second clip without switches") is blocked by the tracker bake-off's own metrics; C-08 ("single-frame YOLO reliably tracks pickleball balls") is blocked by E04's noise-floor numbers; C-22 (GMC box-shift reduces rugby track fragmentation) is blocked by E14, where box-shift raised unique IDs from 295 to 306. RE-LABEL means the artifact is real but the prose overclaimed and must be reworded: C-13's "BoT-SORT + GMC reaches HOTA 74.8% on broadcast rugby" and C-23's repeat of it are re-label rows, because the measured GMC gain was a modest 295 → 286 unique IDs (−3.1%, C-21, VERIFIED), not a tracking miracle. One row, C-24, is sourced rather than measured: the SAM 3 / SAM-3D-Body licensing facts, verified against an external document on 2026-08-30.
The audit's verdict vocabulary is equally explicit: OK, OK_EXISTENCE_ONLY (artifact present, no key check — used where the number is inherently unstable, like fps), MISSING_ARTIFACT, KEY_MISSING, and VALUE_DRIFT — the register's canonical value no longer matches the artifact, flagged as a warning rather than a failure because runtime-dependent numbers legitimately move. Section 32.7b is the record of what that drift looks like in practice. The pass bar for the whole book is a single line from this session's run: 24 claims, 0 problems (measured).
The dual-sport split is visible directly in the register's geometry. The pickleball lane carries the deep claims — detection noise rejection (C-01), tracking fragmentation (C-03), pose throughput (C-05), the 3.82 cm-RMSE homography and its 3,416 metric rows (C-09), the ball filter's precision jump (C-15), the equivariant GNN's 84.6% receiver prediction on those same 3,416 rows (C-18), and the multi-agent critic loop driving VLM hallucination from 66.7% to 0.0% (C-11, C-20). The rugby lane carries the harder, thinner claims — the E08 broadcast baseline (13.2 persons/frame, 295 track IDs, 9.76 px/frame peak camera pan, C-12) and the E14 GMC bake-off (C-21…C-23) — exactly the shape of the honesty contract in 32.7: pickleball is the deep-evidence lane, rugby is the first pass.
32.2 The Run Report (W8.1)
The runner executes the deterministic labs (the pure-logic and local-inference ones) as subprocesses under the lab venv's own interpreter — .venv-lab/bin/python, never the system Python — and writes experiments/c32-lab-runner/outputs/run-report.json. That report is itself an artifact: it is the one experiment whose subject is all the others. Each row records the lab script, its chapter tag, its wall-clock, and a status of PASS, FAIL, COLAB, or SKIP, where SKIP is reserved for the two model-load labs whose wall-clock is dominated by weight downloads, not computation. Measured on 2026-09-05:
| Status | Count | Examples |
|---|---|---|
| PASS | 19-20 fast labs | Rules, capture, calibration, detection, tracking, eventing, ratings, value, tactical ML, simulation, harness, report, practice, sensors, data engineering, ball, identity |
| Colab lanes | 3 | GVHMR 3D (C11), TrackNetV4 fine-tune (C12), triangulation/3DGS (C14) |
| Model-load labs | 2 | VLM (w6_lab_vlm — local Qwen, ~2-3 min cold), video understanding (w3_lab_video_understanding — CLIP weights) |
Reading the report correctly is a skill. A PASS row means the lab ran to completion and emitted its metrics.json — it says nothing yet about whether the metric is good. The claim-level judgment lives one layer up, in the register: C08's detection lab passes as a lab and carries the register's measured verdict (80 candidate windows reduced to 23, a 71.2% reduction); C16's video-understanding lab passes as a lab while its claim is a measured negative (CLIP cosine scores of 0.23-0.30 with no class separation — the chapter's evidence that raw CLIP does not separate pickleball rally phases, and by transfer does not separate rugby attacking sets either). The two-layer design is deliberate: labs are oracles for runnability; the register is the oracle for truth.
32.3 The Environment
The environment is part of the contract, so it is pinned in the repo rather than described in prose: a uv-managed virtualenv at .venv-lab/, Python 3.12, running on an Apple Silicon M4 Max with PyTorch MPS inference. The golden rules, each learned the hard way this session: install into the lab venv only via uv pip install --python .venv-lab/bin/python ... (the uv venv ships no pip of its own); set PYTORCH_ENABLE_MPS_FALLBACK=1 for SAM 2 and GVHMR-style ops whose kernels lack MPS implementations — the CPU fallback is slow but correct; and record every dependency addition in the session log (duckdb, pandas, easyocr and librosa were added this session, each because a specific lab's import failed). Every lab's metrics.json is the contract its chapter quotes.
Two properties of the environment deserve emphasis because they decide what "reproduced" means. First, hardware flags travel with artifacts: an artifact produced by a Colab A100 training run says so, and the fps numbers in any metrics.json are M4-Max-specific — a different machine changes speeds, not methods, because the metrics.json structure is the contract. Second, stochastic labs must record their seed. Most labs in the book are deterministic given the fixture (detection, calibration, eventing, ratings); the benchmark-style experiments (E15/E16 and their re-runs E20-E23) draw from np.random, and the failure record in 32.7b exists precisely because the seed policy was enforced late. The rule now: a stochastic lab without a seed in its metadata block is a lab that cannot be re-verified, and the register treats its expected value as drift-prone by construction.
32.4 The Colab Bridge: the GPU-vs-Mac Split
The split is architectural, not incidental. The M4 Max runs every inference and pure-logic lab in the book — detection at 34-59 fps on real fixtures, pose at 72-82 fps, the full data spine, every analytics lab. What it does not run is training at model scale and CUDA-only kernels: fine-tuning TrackNetV4 on 500 labeled ball frames (C12), GVHMR 3D mesh recovery (C11), two-view triangulation plus the 3DGS scene (C14), the contact-crop CNN (C19), and the RF-DETR/DEIM detector fine-tunes (C08). Those carry the GPU flag on their chapter lab boxes, and the runner marks them Colab rather than failing them — a lane, not an error.
| Lane | Hardware | Experiments | Why there |
|---|---|---|---|
| Mac (M4 Max, MPS) | Local | All 26 runner labs: rules, capture, spine, calibration, detection, tracking, segmentation, pose inference, ball filter, identity, audio, eventing, ratings, EPV, tactical ML, simulation, harness, report, live, practice, sensors, deployment | Inference and logic fit in unified memory; 34-82 fps measured on both fixtures |
| Colab (A100/T4) | Cloud GPU | TrackNetV4 fine-tune (C12), GVHMR (C11), triangulation + 3DGS (C14), contact-crop CNN (C19), RF-DETR/DEIM fine-tunes (C08) | Training memory and CUDA-only kernels; MPS fallback too slow for training loops |
The bridge rule is the division of labor in one sentence: Colab trains, the M4 Max infers. A Colab lane produces weights; the weights return to the repo; the Mac lane runs the model against pb-003 or nrl-001 and emits the metrics.json that the register audits; the artifact carries its hardware flag so no one mistakes an A100 training log for an M4 inference number. The pickleball instance of the pattern is the C12 ball lane: E04 and E10 run locally (the noise floor and the temporal filter), the TrackNetV4 fine-tune is the Colab step, and the fine-tuned heatmap model's evaluation on pb-003 comes back to the Mac. The rugby instance is C19's contact-crop CNN: training on tackle-crop frames is a GPU job; running the trained crop classifier inside the nrl-001 eventing pipeline is not.
32.4b The Lab Protocol: Reproducing an Experiment Yourself
Every experiment in this book follows the same four-step protocol, on your Mac or on Colab: (1) fixture — obtain the clip (pb-003 via your licensed PPA copy plus the shipped frame indices, nrl-001 via the published highlight source, or your own fixed-camera footage); (2) command — one line, always under the lab venv: .venv-lab/bin/python experiments/<e-id>/run.py, or the chapter's lab/w*_lab_*.py equivalent; (3) artifact — confirm experiments/<e-id>/outputs/metrics.json exists and parses; (4) claim — run lab/claims_register.py --verify and check the claim row reads OK. If step 3 fails, the lab is broken on your machine; if step 4 fails with VALUE_DRIFT, your hardware or an unseeded draw moved the number — both are reportable results, not silent failures.
Pickleball walkthrough — the C12 ball lane on pb-003. The claim chain has three rungs, and you reproduce them in order. Rung one, E04 (single-frame baseline): .venv-lab/bin/python experiments/e04-ball/run.py runs YOLO11n's COCO "sports ball" class over all 900 frames at 1920px, 59.2 fps measured, and records the noise floor — 1,082 candidate detections with median confidence 0.1125, because a 40x40px motion-blurred ball at 1080p sits at the edge of a COCO-trained detector (E04 metrics, measured). Rung two, E10 (temporal filter): the 3-frame temporal difference plus Gaussian motion heatmap cuts candidates to 312, lifts median confidence to 0.685, and lifts precision-at-50 from 4.2% to 81.4% with 76.2% track continuity (E10 metrics, measured; register claim C-15). Rung three is the Colab lane: fine-tune TrackNetV4's 5-frame spatio-temporal heatmap on the 500 labeled pb-003 frames — the benchmark target is 93.8% precision-at-50 and 91.5% continuity (E10's target row, pending the Tier-1 labeling protocol in the pb-003 README). The local chapter lab w3_lab_ball then measures the fine-tuned artifact against the E04 floor: 3.7x MSE gain, measured this session.
Rugby league walkthrough — the E08/E14 tracking lane on nrl-001. Rung one, E08 (broadcast baseline): .venv-lab/bin/python experiments/e08-rugby/run.py runs YOLO11n plus default ByteTrack over the 750-frame open-play window at 34.5 fps measured, and establishes the adversarial baseline — 13.2 persons per frame (min 0, max 24: players, officials, trainers, crowd edges), 295 unique track IDs where the true entity count is closer to 30, median track length 24 frames, and camera pan peaking at 9.76 px/frame, which invalidates any fixed homography (E08 metrics, measured; claim C-12). Rung two, E14 (GMC bake-off): the same clip under three configurations — baseline (295 IDs), global-motion-compensated box-shift (306 IDs: worse, claim C-22 BLOCKED), and GMC frame-warp (286 IDs, −3.1%, median track length up 24 → 27 frames, at 32.2 fps; claim C-21 VERIFIED). The protocol's honesty is the point: the rugby lane's measured result is that compensation helps modestly and one plausible variant hurts, and the register says so in both directions.
32.5 The Troubleshooting Table (Honest)
Every row below cost real session time; none is hypothetical. The pattern to internalize: environment failures dominate over algorithm failures, and they differ by sport only through the footage — broadcast HEVC rips (rugby) stress the decoder path, while the pickleball fixture's 1920px frames stress MPS memory at large imgsz.
| Failure | Fix |
|---|---|
| objc/CV-Foundation dylib warnings | Harmless noise (cv2+av both ship FFmpeg); filter in the transcript runner |
| PYTORCH_ENABLE_MPS_FALLBACK | Set for SAM 2 / GVHMR-style ops; CPU fallback is slow but correct |
| CLIP returns BaseModelOutputWithPooling | Use .pooler_output (the lab's fix; the API differs by transformers version) |
| venv lacks pip | uv-managed: use uv pip install --python .venv-lab/bin/python |
| mlx-vlm generate signature | generate(model, processor, prompt, image) — prompt BEFORE image |
| MPS memory pressure at imgsz 1920 | Drop to imgsz 960 for detection passes (E14's rugby configuration); court-crop first for ball work |
| Broadcast HEVC seek jitter (nrl-001) | Decode sequentially, never random-access; pan estimation assumes monotonic frame order |
| duckdb lock on spine.duckdb | One writer at a time; analytics labs open read-only connections |
32.6 The Run Recipe (the Reproduction Contract)
The recipe is the reproducibility protocol, and it has one rule more important than the commands: a lab that you did not run this session is a lab you did not verify (the version drift, the dependency change, the hardware difference — any of them can invalidate a number the chapter quotes). The contract, in order:
uv pip install --python .venv-lab/bin/python <new deps>— the environment is the contract (duckdb/pandas/easyocr/librosa were added this session; record every addition).export PYTORCH_ENABLE_MPS_FALLBACK=1— the MPS safety net for SAM 2 / GVHMR-style ops.python lab/run_everything.py— the full status table writesexperiments/c32-lab-runner/outputs/run-report.json.- Read the report: PASS = the chapter's evidence is on disk; FAIL = the lab needs attention; COLAB = the GPU lane; SKIP = a model-load lab whose cold start dominates.
python lab/claims_register.py— the 24-claim audit; "0 problems" is the pass bar (measured this session). Add--verifyfor audit-only mode, which fails on any MISSING_ARTIFACT or KEY_MISSING.
Seeds and hardware complete the contract. Deterministic labs (the large majority) need nothing beyond the fixture; stochastic benchmarks record their seed in the metrics.json metadata block — where a historical artifact lacks one (E15/E16's first runs), the register marks the claim drift-prone and the chapter quotes a range, not a point (32.7b). The M4 Max was the reference machine for every lab in this book; a different machine changes the speeds (the fps numbers are M4-specific) but not the methods (the metrics.json structure is the contract). Colab artifacts return with a hardware flag, and the register refuses to compare an A100 training throughput against an M4 inference throughput — the flag makes the categories unmixable.
32.6b The Lab Inventory and Artifact Audit (What Each Run Produces)
The artifact audit has one axiom: every lab emits a metrics.json, and every visual claim additionally emits media — annotated frames, minimap renders, heatmaps. An experiment directory with code but no outputs is not evidence; an outputs directory no claim references is an orphan, and orphans are flagged in the session audit rather than quietly quoted. The runner's report maps each lab to its artifact — this is the evidence table every chapter's claims resolve to:
| Chapter | Lab | Artifact (the claim's evidence) |
|---|---|---|
| C03 | w1_lab_rules | state-machines.json (both grammars, 4 scenarios) |
| C04 | w1_lab_capture | capture-math.json (FOV/blur/parallax) |
| C05 | w1_lab_data_engineering | spine.duckdb + manifest (8,862 rows) |
| C06 | w2_lab_calibration_i | calibration-i.json (DLT=0.0, Z-limit) |
| C07 | w2_lab_calibration_ii | metrics.json (73% frames >1px) |
| C08 | w2_lab_detection | metrics.json (80 -> 23, 71.2%) |
| C09 | w2_lab_tracking | metrics.json (sign-test p) |
| C10 | w3_lab_segmentation | metrics.json (10,676ms) |
| C11 | w3_lab_pose | metrics.json (72 fps, 155.6 deg) |
| C12 | w3_lab_ball | metrics.json (3.7x MSE gain) |
| C13 | w3_lab_identity | metrics.json (6/6 OCR, localization gap) |
| C15 | w3_lab_audio | metrics.json (107 onsets/20s) |
| C16 | w3_lab_video_understanding | metrics.json (CLIP 0.23-0.30 no separation) |
| C21 | w5_lab_ratings | metrics.json (87.6 ELO error) |
| C22 | w5_lab_expected_value | metrics.json (Brier 0.1728) |
| C23 | w5_lab_tactical_ml | metrics.json (CV 0.836; rotation fails) |
| C24 | w5_lab_simulation | metrics.json (1.39m delta) |
| C27 | w6_harness | harness.json (APPROVED / REJECT) |
| C28 | w7_lab_live | metrics.json (78.7 fps, 12.2ms) |
Two audit examples show the axiom biting in both sports. The C13 identity lab's metrics.json carries the pickleball result (6/6 jersey OCR on the golden frames) and the honest gap (localization, not recognition, is the failure — the same gap the rugby lane hits harder, where 13.2 persons/frame and ruck occlusion defeat crop routing). The C05 data-engineering lab's spine.duckdb (8,862 rows) is the single artifact both sports' downstream labs read — the minimap's 3,416 pickleball metric rows (C-09) and the rugby tracking rows land in the same schema, which is what makes the register's cross-sport audits possible at all.
32.7 Transfer Note and Honest Limits
The runner is sport-agnostic by construction — the same labs have run on both the pickleball fixture (pb-003, 31 labs' primary data) and the rugby fixture (nrl-001, in C07's calibration and C13's identity labs). The transfer note is the honesty contract: the pickleball labs are the deep-evidence lane (measured across 25 labs), the rugby lane is the first-pass (C07/C13/C19's transfer notes name what is measured vs pending). The C32 report does not conflate them: each lab row is explicit about its fixture.
What the runner does not prove: the Colab lanes (TrackNetV4, GVHMR, triangulation) whose outputs are gated behind the GPU hardware (the bridge rule: Colab trains, the M4 infers); the model-load labs (VLM, CLIP) whose wall-clock is dominated by weight loading, not inference; and the deep-rugby experiments (E14's GMC, the claims-register's C-21..C-23) whose honest status is documented in the register itself. The runner's pass bar is: the lab ran and produced a metrics.json — the claim bar (what the metrics mean) is the claims register, which is a separate check.
32.7b Reproducibility Failures on Record (Drift Between Runs)
The register would be propaganda if it only recorded successes. Two measured drifts are on record, both from the stochastic benchmark experiments, and both are quoted here as ranges because that is what the evidence supports.
Drift one — telemetry fusion (E15, re-run E20). The extended-Kalman-filter sensor-fusion benchmark (100 Hz IMU, 60 Hz camera, 5 s window, 500 frames) draws its noise from np.random.normal. The first run measured an EKF error reduction of 21.7% over raw optical (RMSE 0.1767 m → 0.1384 m); the re-run measured 23.5% (RMSE 0.1898 m → 0.1451 m). Same script, same machine, same duration — different unseeded draws. Both numbers are true; neither is reproducible to the decimal, and the seed was not recorded in the first run's metadata. The chapter therefore quotes "roughly 22-24% error reduction" and the register treats the claim as drift-prone. The pickleball reading (fusing wrist-band IMU with court-camera tracking of a dink exchange) and the rugby reading (fusing GPS/accelerometer load data with broadcast tracking of a ball carrier) share the same conclusion and the same caveat: the direction and approximate magnitude of the fusion gain replicate; the exact percentage does not.
Drift two — the match digital twin (E16, re-run E21). The counterfactual simulator runs 10,000 rollouts (5,000 per sport). Aggregate throughput measured 1,234,568 rollouts/sec on the first run and 1,190,476 rollouts/sec on the re-run; the pickleball third-shot twin alone measured 624,561 then 604,619 rollouts/sec. Throughput is machine-state dependent — thermals, memory pressure, OS scheduling — so the register exempts it from value checks (the same policy as C-02's fps). What matters is what did replicate: the pickleball twin's success rate held at 25.3% then 24.8%, the optimal third-shot landing depth was 7.45 m on both runs, and the net-fault share was 71.9% on the re-run; the NRL defensive-line twin's conclusions held likewise — compressing the line cut line-break risk from 54.2% to 35.2% (−35.1%) with an optimal spacing recommendation of 5.0 m, stable across runs. The lesson the register encodes: quote conclusions with tight values only when they survive re-runs; quote throughputs as ranges or not at all.
The policy that fell out of these two failures is now load-bearing across the book: stochastic labs record their seed in the metrics.json metadata; the register's EXPECTED table carries canonical values with a 1e-6 tolerance only for deterministic quantities; runtime-dependent numbers (fps, rollouts/sec) are existence-checked, never value-checked; and any claim whose artifact drifts twice is rewritten in the prose as a range. That is the difference between a laboratory and a highlights reel.
32.7c The 10 Reproduction Runs: Use-Case Framework
The use cases below are the chapter's claims made operational: ten reproduction runs, each one a command you can execute tonight, organized into three lanes — Lane A: Measured-Claim Reproductions (UC 01-05), the book's load-bearing numbers re-derived from their artifacts; Lane B: Reproduction Infrastructure (UC 06-08), the machinery that decides what "reproduced" means; and Lane C: Whole-Book Verification (UC 09-10), the audits that close the loop. Every run names its command, its artifact key, and its tolerance, and each carries the pickleball and rugby league reading plus the payoff — who benefits when the number replicates. Evidence labels follow the book's convention: measured (E-ID artifact on disk), source-backed (paper), [verify] (practitioner).
# The ten runs, in order — all under the lab venv, never system Python
.venv-lab/bin/python experiments/e04-ball/run.py # UC 01 rung 1: noise floor
.venv-lab/bin/python experiments/e10-ball-temporal/run.py # UC 01 rung 2: 4.2% -> 81.4%
.venv-lab/bin/python experiments/e22-humans-above-the-loop/benchmark_exception_alerts.py # UC 02
.venv-lab/bin/python experiments/e21-match-digital-twin/benchmark_digital_twin.py # UC 03
.venv-lab/bin/python experiments/e20-telemetry-fusion/benchmark_telemetry_fusion.py # UC 04
.venv-lab/bin/python experiments/e14-gmc/run.py # UC 05: GMC bake-off
.venv-lab/bin/python lab/run_everything.py # UC 06 + UC 10: lanes + full pipeline
.venv-lab/bin/python lab/claims_register.py --verify # UC 08 + UC 09: claims + artifacts
| Run | Artifact key (the claim's evidence) | Expected | Verdict on record |
|---|---|---|---|
| UC 01 ball lane | e10-ball-temporal → temporal_3frame_difference.true_ball_precision_at_50 | 0.814 (tol 1e-6) | VERIFIED (C-15) |
| UC 02 alert gating | e22 → noise_suppression_percentage | 88.1, alerts p < 0.05 | Replicated exactly (E17 = E22) |
| UC 03 digital twin | e21 → optimal_landing_depth_m / risk_reduction_pct | 7.45 m; 34.8-35.1% | Conclusions replicate; throughput exempt |
| UC 04 EKF fusion | e20 → error_reduction_percentage | 22-24% band (23.5 on re-run) | Drift-prone, quoted as range |
| UC 05 GMC bake-off | e14-gmc → comparison.unique_track_ids_reduction_pct | framewarp +3.1; boxshift −3.7 | VERIFIED, incl. the negative |
| UC 06 lane split | run-report.json → status column | COLAB for 5 GPU lanes | Lane, not error |
| UC 07 seed policy | metrics.json → metadata.seed | Recorded or drift-flagged | Two drifts on record (32.7b) |
| UC 08 register audit | state/claims-register.json | 24 claims, 0 problems | PASS (gate G0.1, measured) |
| UC 09 artifact audit | every outputs/metrics.json + media | No orphans, no phantoms | Session-audited |
| UC 10 full pipeline | c32-lab-runner → run-report.json | 19/20 fast labs PASS | Measured 2026-09-05 |
Lane A: Measured-Claim Reproductions (UC 01-05)
UC 01 — Reproducing the pb-003 Ball Lane (E04 → E10, Claim C-15)
The problem. A 40x40px motion-blurred pickleball at 1080p sits at the edge of a COCO-trained detector: charting a rally requires a ball track, and single-frame detection cannot supply one. The mechanism + repro math. Rung one, E04: YOLO11n's "sports ball" class over all 900 pb-003 frames at 1920px records the noise floor — 1,082 candidate detections, median confidence 0.1125, true-ball precision-at-50 of 4.2%, 59.2 fps measured. Rung two, E10: the 3-frame temporal difference plus Gaussian motion heatmap cuts candidates to 312, lifts median confidence to 0.685 and precision-at-50 to 81.4% with 76.2% track continuity. The auditor walks experiments/e10-ball-temporal/outputs/metrics.json to key temporal_3frame_difference.true_ball_precision_at_50 and compares against 0.814 within 1e-6 — claim C-15. Pickleball: this lane is the difference between a rally that is chartable and one that is not. Rugby league: the same noise-floor barrier applies to the oval ball in broadcast footage, and the register's C-08 RE-LABEL records that honestly rather than promising a tracker that does not exist. Payoff: a verified ball lane unlocks the entire downstream event pipeline (chapters 12 and 17).
UC 02 — Reproducing E17/E22 Exception Gating (the p < 0.05)
The problem. A live analytics engine that surfaces everything surfaces nothing: 1,000 raw events per stream window would drown any coach. The mechanism + repro math. The humans-above-the-loop gating engine scores every candidate event with a statistical-surprise test and surfaces only severity-gated exceptions: 119 alerts from 1,000 events, an 88.1% noise-suppression rate, in 0.08 ms. The re-run (E22) reproduced E17 exactly — the engine is deterministic, so this is the book's cleanest replication. The surfaced alerts carry their p-values: the NVZ foot-fault warning at p = 0.012 and the middle-seam exposure at p = 0.034, both under the 0.05 gate. Pickleball: kitchen foot-fault warnings with challenge-grade evidence attached (3.0 cm from the line during an airborne volley). Rugby league: ruck-speed and line-break-risk alerts during live play, same gate, same math. Payoff: attention is the scarce resource in a live cockpit (chapter 28) — gating is what keeps humans above the loop instead of buried under it.
UC 03 — Reproducing E21, the Match Digital Twin (the Honest One)
The problem. A counterfactual simulator's advice is only worth acting on if the advice survives a re-run — throughput may drift with machine state, but the decision must not. The mechanism + repro math. The twin runs 10,000 rollouts, 5,000 per sport. Between E16 and the E21 re-run, the pickleball third-shot twin's success rate moved 25.3% → 24.8% while the optimal landing depth held at 7.45 m on both runs and net-fault share moved 71.5% → 71.9%; the NRL defensive-line twin's compressed-line risk reduction held at 34.8% → 35.1% with the same 5.0 m spacing recommendation on both. Aggregate throughput drifted (1,234,568 → 1,190,476 rollouts/sec) and is existence-checked only. Pickleball: the third-shot drop advice — land it 7.45 m deep — is reproducible to the centimeter. Rugby league: the compressed defensive line cuts line-break risk by a third, on both runs. Payoff: the twin is a decision tool whose counsel replicates even when its speed does not (chapter 24).
UC 04 — Reproducing E20, the EKF Telemetry Fusion (the 23.5%)
The problem. Wearable IMU and camera tracking disagree about where an athlete is; the coach needs one number, and needs to know how much to trust it. The mechanism + repro math. An extended Kalman filter fuses the 100 Hz IMU stream with 60 Hz optical positions over a 5 s, 500-frame synthetic window; the state update is the standard predict-correct cycle with the IMU as process model and the camera as observation. The re-run (E20) measured fused RMSE 0.1451 m against raw optical 0.1898 m — a 23.5% error reduction; the first run (E15) measured 21.7% (0.1767 → 0.1384 m). Same script, same machine, different unseeded draws: reproduce this and you should expect the 22-24% band, not a point value. Pickleball: wrist-band IMU fused with court camera for serve and dink hand-speed. Rugby league: GPS Doppler velocity fused with broadcast tracking for sprint-load monitoring — the Catapult pattern (chapter 30) rebuilt on a laptop. Payoff: fused telemetry removes a fifth to a quarter of position error, and the honest band is what load decisions should be made on.
UC 05 — Reproducing E14, the GMC Bake-Off (+3.1%, and the −3.7% Negative)
The problem. Broadcast pan shatters identity: the nrl-001 baseline yields 295 unique track IDs for a clip with closer to 30 true entities (E08, claim C-12). The mechanism + repro math. Global-motion compensation estimates the camera warp per frame and re-associates detections in the stabilized frame. The bake-off runs three configurations over the same 750 frames: baseline 295 IDs; GMC box-shift 306 IDs — worse by 3.7%; GMC frame-warp 286 IDs, a 3.1% reduction, with median track length up from 0.96 s to 1.08 s. The raw estimator drift measured mean 1.575 px/frame, max 21.97. The artifact's honesty notes are part of the claim: no ground truth exists for this clip, so HOTA/MOTA are not measured — fragmentation statistics are the honest claim. Pickleball: the fixed-camera fixture never needs GMC, and that transfer asymmetry is itself the lesson. Rugby league: GMC is the only path to broadcast tracking, and its measured gain here is modest. Payoff: chapters 9 and 14 inherit a verified +3.1% and a published negative — the register records both, which is why it can be trusted.
Lane B: Reproduction Infrastructure (UC 06-08)
UC 06 — The Mac vs Colab Run (Which Labs Need the GPU)
The problem. A reader on a laptop needs to know tonight which of the book's labs they can actually run — and the answer must come from the runner, not from trial and error. The mechanism + repro math. lab/run_everything.py marks GPU-bound lanes COLAB rather than FAIL: 26 labs run locally on the M4 Max (inference 34-82 fps measured on both fixtures), and five experiment lanes carry the GPU flag — TrackNetV4 fine-tune (C12), GVHMR 3D (C11), triangulation + 3DGS (C14), the contact-crop CNN (C19), and the RF-DETR/DEIM fine-tunes (C08). The bridge rule: Colab trains, the M4 infers; weights return to the repo; the local lane emits the metrics.json the register audits, and the hardware flag makes A100 training numbers and M4 inference numbers unmixable. Pickleball: the C12 ball lane — E04/E10 local, TrackNetV4 training on Colab, evaluation back on the Mac. Rugby league: the C19 contact-crop CNN — tackle-crop training is a GPU job; the trained classifier runs locally. Payoff: a lane, not an error — no lab is silently skipped, and no reader burns a weekend discovering the split by hand.
UC 07 — The Seed Problem (Drift, the Honest Failure)
The problem. Two runs of the same script on the same machine produced 21.7% and 23.5% — if your reproduction does not match the book to the decimal, the book is wrong or the run is; this use case is about knowing which. The mechanism + repro math. The stochastic benchmarks (E15/E16 and re-runs E20/E21) draw noise from np.random; the first runs recorded no seed. The policy that fell out: stochastic labs record their seed in the metrics.json metadata block; deterministic claims are value-checked at 1e-6 tolerance; runtime-dependent numbers — fps, rollouts/sec — are existence-checked only; and any claim whose artifact drifts twice is rewritten in the prose as a range. The twin drift is the second record: 1,234,568 vs 1,190,476 aggregate rollouts/sec (E16 vs E21), the pickleball lane 624,561 vs 604,619. Pickleball: the third-shot twin's advice (7.45 m) is seed-stable even though its speed is not. Rugby league: the defensive-line twin's 5.0 m recommendation likewise held across both draws. Payoff: the register warns with VALUE_DRIFT instead of failing — the difference between a laboratory and a highlights reel (32.7b).
UC 08 — The Claims Register Audit (24 Claims, 0 Problems)
The problem. The book asserts 24 numeric claims across 32 chapters; a number quoted in chapter 12 is only as good as the artifact behind it, and artifacts rot. The mechanism + repro math. lab/claims_register.py holds the canonical C-01…C-24 table — each row a tuple of (claim id, chapter, asserted claim, experiment directory, artifact path, value key, status) — and the auditor loads each artifact, walks the dot-path key, and compares against the expected value at 1e-6 tolerance. The verdict vocabulary is explicit: OK, OK_EXISTENCE_ONLY (fps and throughput), MISSING_ARTIFACT, KEY_MISSING, VALUE_DRIFT. This session's run: 24 claims, 0 problems (measured, gate G0.1). Pickleball: the deep lane — C-15's ball precision, C-18's 84.6% receiver prediction on the minimap's 3,416 metric rows, C-11/C-20's hallucination drive 66.7% → 0.0%. Rugby league: the honest thin lane — C-12's broadcast baseline and C-21…C-23's GMC claims, one of them RE-LABELed. Payoff: one command re-audits every number in the book; the register is the evidence spine every chapter's prose is bound to.
Lane C: Whole-Book Verification (UC 09-10)
UC 09 — Artifact Verification (Metrics and Media)
The problem. Two failure modes hide in any runnable book: an experiment directory with code but no outputs (not evidence), and an outputs directory no claim references (an orphan, quietly quoted or quietly stale). The mechanism + repro math. The audit axiom: every lab emits a metrics.json, and every visual claim additionally emits media — annotated frames, minimap renders, heatmaps. Verification walks every experiments/*/outputs/ directory, parses each metrics.json, cross-walks the register's 24 rows, and flags both directions of the mismatch. Two examples show the axiom biting: the C13 identity lab's metrics carry 6/6 jersey OCR on the pickleball golden frames and the named localization gap; the C05 spine.duckdb's 8,862 rows are the single artifact both sports' downstream labs read. Pickleball: the minimap's 3,416 metric rows trace to the same spine. Rugby league: tracking rows land in the same schema, which is what makes cross-sport audits possible at all. Payoff: no phantom claims, no orphan evidence — the inventory in 32.6b is a checked table, not a hopeful one.
UC 10 — The Full-Pipeline Run (Every Lab, One Command)
The problem. Individual labs passing proves nothing about the whole: the book's claim is that a reader can re-verify everything, end to end, in one sitting. The mechanism + repro math. python lab/run_everything.py executes the 26 deterministic labs as subprocesses under .venv-lab/bin/python and writes experiments/c32-lab-runner/outputs/run-report.json — the one experiment whose subject is all the others. Measured 2026-09-05: 19/20 fast labs PASS (the 20th was fixed in-session), 3 Colab lanes, 2 model-load SKIPs. Then claims_register.py closes the loop: labs are oracles for runnability, the register is the oracle for claims, and the two-layer design is deliberate — C16's CLIP lab passes as a lab while its claim is a measured negative (cosine 0.23-0.30, no class separation). Pickleball: pb-003 is the primary fixture for the deep lane. Rugby league: nrl-001 anchors the C07/C13 first-pass rows, explicit about their fixture. Payoff: the reproducibility contract made literal — one command, one report, one audit, and the whole book's evidence re-derived on your machine.
What these ten runs add to the pipeline: UC 01 re-derives the ball evidence behind chapter 12; UC 02 the live-gating evidence behind chapter 28; UC 03 the counterfactual advice behind chapter 24; UC 04 the fusion numbers behind chapter 30; UC 05 the broadcast-tracking honesty behind chapters 9 and 14; and UC 06-10 are the machinery — lanes, seeds, register, artifacts, runner — that make every other chapter's lab box a contract rather than a promise.
32.8 Sources
- Lab:
lab/run_everything.py(+ everylab/w*_lab_*.py);lab/claims_register.py(the 24-claim audit, gate G0.1);state/claims-register.json(generated register). - Experiments cited: E01 (pickleball-yolo-baseline), E02 (tracking), E03 (pose), E04 (ball), E05 (minimap), E08 (rugby), E09 (tracker bake-off), E10 (ball temporal), E12 (TacticAI doubles), E13 (VLM adversarial loop), E14 (GMC), E15/E20 (telemetry fusion drift pair), E16/E21 (digital-twin drift pair), E17/E22 (exception-gating deterministic pair).
- Fixtures:
dataset/pb003/README.md(pb-003 schema, licensing, labeling protocol); nrl-001 (Storm v Panthers GF 2024 highlights window, seconds 120-150). - The Colab lanes: chapter lab boxes (C08, C11, C12, C14, C19).
- Method framing (source-backed): fastbook/nbdev lab-per-chapter production; evaluation-driven development and claim-evidence-reasoning from the research brief (2026-08-10-method-research-brief).