AS '26
All Chapters

Tagging · SECTION 13

Chapter 13 — Identity: Who Is Who

Track IDs are not people: jersey OCR, ReID embeddings, team clustering, roster priors, and the consent line

Reading time

35 min

13 Chapter 13 — Identity: Who Is Who

Track IDs are not people: jersey OCR, ReID embeddings, team clustering, roster priors, and the consent line

13.1 The Identity Problem

Detection finds people. Tracking keeps boxes in a sequence. Identity answers who they are — and it is the hardest of the three. A track ID is a hypothesis about persistence, not a person: chapter 9's E02 produced 26 track IDs for 4 pickleball players, and E08's broadcast rugby clip spawned 295 track IDs for roughly 30 humans — 286 even after E14's global-motion-compensation repair. The tracker has no concept of "Tedesco" or "Anna Leigh Waters"; it has "track 147."

Three verbs the literature conflates, and this book keeps separate. Detection — where is a person in this frame (chapter 8); a solved purchasing decision for players. Tracking — which detection in the next frame is the same detection (chapter 9); track IDs are spatiotemporal continuations. Identity — which named, roster-listed person this track belongs to. Identity is semantic binding: it requires evidence the tracker never sees — a number, a kit, a role, a lineup sheet — and it must survive everything tracking cannot: camera cuts, a player off-frame for a full set, half-time, kit clashes, and the broadcast director.

Three consequences make this its own chapter. First, downstream stats are identity-typed, not track-typed: "player 6's running metres" is garbage if track 147 is 60% player 6 and 40% player 7. Identity-mixing inside a single track is the silent failure — no standard MOT metric counts it. Second, identity evidence is episodic: a rugby jersey number is legible in perhaps 5% of frames; the pickleball serve-side rule pins identity only at serve time. The system must bind when it can and propagate when it cannot. Third, identity is audience-dependent: a pro club has a roster, numbered kits, and wearable ground truth; a social pickleball player has four friends and a phone camera. There is no universal identity stack — §13.9 is the decision table this chapter serves.

The honest frame: identity is a fusion problem, not a single-model problem. The signals available: appearance (ReID), jersey numbers (OCR), team color (clustering), role/side (the sport's structure), roster (external data), and — the one to avoid for amateurs — faces. The design rule that governs everything below: declarative signals create identity; probabilistic signals only propagate it.

13.2 The Signal Table

Signal Strength Weakness Use Case
Appearance ReID Works without numbers; per-session gallery Fails on same-kit teammates; lighting drift Pickleball doubles (4 people, distinct)
Jersey OCR Explicit identity; durable across views Needs localization (which region is a jersey); low-res broadcast Rugby (13 same-color players)
Team clustering Cheap; separates teams instantly Fails when kits are similar (all-black pickleball) First split before identity
Role/side priors Zero-cost; the sport's structure Only valid for structured moments (serve, formation) Doubles serve order, rugby position
Roster fusion External ground truth joins Needs a roster source; pro-only Professional team infrastructure
Face Strongest identifier Ethics: consent, biometric law (C31) Never for amateurs without explicit consent

Read the table as a fusion order, not a menu. In rugby league the stack runs: team clustering halves the search space → jersey OCR declares a number where legible → the roster constrains that number to who is actually on the field → position priors sanity-check it → ReID propagates the binding through the tackle and the ruck until the next declarative read. In pickleball the stack is shorter and stranger: no numbers exist, kits routinely clash, so gallery enrollment plus the serve-side rule do the declaring, and ReID does all the propagating. Same fusion shape, completely different evidence.

Identity resolution architecture: three evidence lanes jersey OCR team clustering rules+roster dropping evidence onto a track timeline with confidence band. Recreate: white background, burnt-orange discs.
Figure 13.1: Identity As Fusion. Three evidence lanes bind to a track timeline; the confidence band dips at stacking windows and cuts, then re-pins. No single model resolves identity.

13.2b The Transient Identity Problem and the Registry

Identity in sport is transient: the mapping between bodies and names changes underneath the system while it runs. Rugby league rotates its 14–17 bench through interchange churn, sin-bins, and blood bins — the active set of 13 changes minute by minute. Kits change at half-time in social footage and between matches in tournaments. Broadcast coverage cuts between the wide tactical camera, the close-up, and the replay, resetting tracklets faster than any vote can accumulate — E08's 295 spawned IDs on nrl-001 are mostly the director's fault, not the tracker's. Even pickleball has its version: sides switch at 11 in a game to 21, stacking moves players across the net line mid-rally, and a player walking off to fetch a ball re-enters as a fresh detection.

The data structure that absorbs all of this is the identity registry: a table mapping (track_id, t_start, t_end) → identity, confidence, provenance. Two event types write to it. Binding events are declarative: a legible jersey read, a serve-side deduction, an enrollment match — each attaches a name to a track interval with a provenance tag. Propagation intervals are probabilistic: ReID similarity and motion continuity carry the last binding forward, with confidence decaying as a function of time-since-binding, occlusion density, and kit ambiguity. A broadcast cut or a detected substitution is a registry boundary: propagation stops, the interval closes, and the system waits for the next binding event instead of guessing. Association through tracking is what fills the intervals: the tracker hands the registry a continuous spine of boxes, and the registry's job is to attach names to segments of that spine — never to extend a name past the spine's breaks.

Confidence calibration is the product. A registry that says "track 147 is player 9, confidence 0.91, bound by jersey read at 54:12, propagated 210 frames" is usable; one that emits bare names is a liability. The discipline mirrors chapter 20's: every identity claim carries its provenance (which module bound it) and every downstream stat joins against intervals, not bare track IDs. The silent compound failure — a hard prior plus no abstention channel producing a confident wrong name — is the one failure mode this chapter prohibits by construction.

Identity pipeline: detection boxes on a pickleball player and rugby player, embedding extraction, cosine matching against gallery, identity registry with confidence bars. Recreate: white background, black linework, burnt-orange stage labels.
Figure 13.2: The Identity Pipeline. Detect → embed → match → register. The registry is the deliverable: named intervals with confidence and provenance, not bare track integers.

13.3 Jersey OCR: The Measured Reality (W3.5 Lab)

Our lab ran digit-only OCR (EasyOCR, CPU, digits allowlist) on 6 nrl-001 broadcast frames (measured, C13-lab):

Result Value
Frames analyzed 6
Frames with digit reads 6 (every frame — OCR is not the problem)
Digit regions per frame 6–13
What was read "62114" (scoreboard), "5", "12", "17", "343", "41485" (conf 0.32–1.0)
Confidence distribution 0.32 to 1.00 — irrelevant digits pollute the reads

The verdict is clear: OCR capability is not the bottleneck — localization is. Digits appear on scoreboards, advertising hoardings, and crowd signage at the same confidence as jersey numbers; the highest-confidence reads in the lab ("6" at 1.0, "4" at 1.0) are not on players. The evidence says jersey identity is load-bearing where it works: in the SoccerNet GSR ablation (source-backed, arXiv:2404.11335), jersey is the single most damaging identity attribute — image-only HOTA 57.64 steps down through pitch localization (42.65), role (40.76), and team (37.03) to 25.65 with jersey added, and 22.26 for the full GS-HOTA task. Adding the jersey requirement costs more than calibration, role, and team combined. The sn-jersey tracklet benchmark tops at 92.85% against a 3.93% random baseline — and the winning systems win by knowing when not to read: its ground truth is one number per tracklet with "-1" for never-visible, so abstention is a first-class output. No public rugby or pickleball jersey dataset exists (searched 2026-09-01) — this book's jersey-OCR lab is the first rugby baseline of its kind.

The fix is pipeline, not model: detect persons (chapter 8) → crop the upper torso (optionally pose-guided — E03's pose stack runs 82 fps on M4 and puts the read region on the back torso, not the whole box) → legibility gate → OCR inside the crop → digit-regex filter → tracklet majority vote. The vote is the load-bearing trick: per-frame reads are noise (motion blur, folds, occlusion), but the mode over 100–300 frames is signal. The GSR reference stack is YOLOv8 + StrongSORT + PRTreID + MMOCR detect/read + KMeans team affiliation, with abstain when illegible. Koshkina's CVPRW 2024 pipeline (legibility classifier → pose-guided RoI → PARSeq → tracklet consolidation, with a ReID outlier step that drops mis-associated crops before they poison the vote) is the clean reference — but CC BY-NC 3.0: reproduce to learn, rebuild with PARSeq (Apache-2.0) for shipping.

Rugby league is friendlier to jersey OCR than soccer: large back numbers 1–17, a smaller front number beside the crest, and broadcast close-ups after every try. The hostility comes from the ruck (numbers buried under bodies), mud, jersey deformation in the tackle, and cuts resetting tracklets. Pickleball has no numbered jerseys at all — this section simply does not transfer; §13.3b–§13.5 carry pickleball identity.

Rugby league jersey OCR: rear view of player wearing number 13, magnified torso crop with digit boxes, tracklet vote tally resolving to 13. Recreate: white background, black linework, burnt-orange accents.
Figure 13.3: Jersey OCR On The Back. Crop the torso, read the digits, vote across the tracklet. The struck-through frames are the majority — the system wins by abstaining on them.

13.3b ReID Embeddings: Propagation, Not Declaration

Appearance re-identification turns a player crop into a vector — 512 floats for OSNet, 2048 for ResNet50-ibn — such that crops of the same body land near each other under cosine distance. Chapter 9 owns the mechanics (OSNet-512 as the M4-native workhorse; the EMA-gallery drift-capture failure; the same-kit embedding collapse); this chapter owns what ReID can and cannot mean for identity. An embedding match says "probably the same body shape and kit." It cannot distinguish twin teammates in identical kit, and it has no concept of a name. The book's rule: ReID propagates identity between binding events; it almost never creates it.

The upgrade that turns ReID from a similarity tool into an identity system is gallery enrollment — the amateur answer. Ten seconds of each known player alone in frame at session start builds a per-session gallery of named embeddings; every subsequent track is assigned to the nearest enrolled identity by cosine similarity with a rejection threshold, instead of to an anonymous integer. For pickleball doubles this is the entire declarative layer before the serve rule kicks in: four players, four cards, one 10-second walk-through each. For rugby it is nearly useless at match scale — 30 athletes in two identical kits put 15 near-duplicate bodies into one embedding neighborhood, which is exactly the same-kit collapse chapter 9 measured.

The sport-correct architecture is multi-task: PRTreID (the GSR baseline's ReID, source-backed) trains one backbone jointly for identity, team affiliation, and role — triplet losses on identity and team, focal loss on a role head — so the tracker, the team clustering, and the role priors all read structure out of the same vector. Single-purpose street ReID trained on Market-1501 is the bootstrap, not the destination. And the honest failure note: embeddings drift with lighting — a shadow band across the court moves every embedding mid-clip — so galleries update under an EMA with updates frozen during ambiguity, and every propagation interval re-pins at the next binding event rather than accumulating similarity indefinitely.

ReID embedding space: four separated pickleball player clusters with centroids and a query point with cosine arrows, plus two merged same-kit rugby teammate clusters. Recreate: white background, black linework, burnt-orange accents.
Figure 13.4: The Embedding Space. Four enrolled pickleball players separate cleanly; two rugby teammates in identical kits merge into one neighborhood. Distance propagates identity — it cannot declare it.

13.4 Team Clustering: The Football_Analysis Pattern

Before who, which side. Team affiliation halves the identity search space and gates every downstream prior. The transferable recipe comes in two generations. Classic color clustering: crop each player box, take the torso pixels, k-means them into 2 (or 3, with the referee) clusters in HSV space — zero training data, runs anywhere. Embedding clustering (roboflow/sports, MIT): SigLIP image embeddings of player crops → UMAP reduction → KMeans with k=2; semantic features survive shadows and sponsor clutter that break raw-color k-means. The GSR baseline does the learned version: average PRTreID embeddings over each tracklet, KMeans into two, label clusters left/right from mean pitch position.

It works when kits differ and collapses when they don't — and the collapse has three distinct mechanisms, all observed in this book's footage. Similar kits merge the clusters: pickleball's social reality — four players in dark athletic wear — puts everyone in one blob regardless of color space or embedding model; k=2 must be abandoned, not forced. Lighting splits one team in two: a shadow band across the pitch makes one team's centroids straddle. Non-players contaminate k=2: referees, ball kids, and the E01-style bench/crowd leak must be filtered before clustering (the court-polygon filter took E01 from 69 boxes to 11), or the clusters describe "players vs officials," not team vs team. On nrl-001 the kits are distinct and both generations work; on pb-003 they aren't and neither does — that asymmetry is the point.

Two disciplines keep clustering honest. Cluster identity is per-crop and flickers, so it gets the same tracklet majority-vote treatment as jersey numbers — a per-frame label never overwrites a confident tracklet-level assignment. And when the inputs declare themselves unclusterable (the all-black case), the system says so and falls through to enrollment and role priors instead of drawing an arbitrary boundary through one blob and believing it.

Team clustering comparison: two clean rugby kit clusters with decision boundary versus one merged all-black pickleball blob with a crossed-out forced split. Recreate: white background, black linework, burnt-orange accents.
Figure 13.5: Same Algorithm, Two Realities. Distinct rugby kits split cleanly; all-black pickleball collapses to one blob. The forced k=2 boundary is arbitrary — abandon it, don't trust it.

13.5 The Role-Prior Identity (Pickleball Doubles)

From chapter 9, applied — and for a 4-player court this is the tractable identity path. Pickleball doubles is a determinate identity problem at serve time. Four known people, a fixed serving order, score parity pinning the server's court half (even score: right court; odd: left), and a net plane no feet cross: at each serve, at least two of the four identities — server and receiver — are deduced, not inferred, and the partners follow by elimination. With E05/E11-grade homography (3.82–4.65 cm RMSE) mapping feet to court coordinates, this is arithmetic, not machine learning.

The full mechanism runs as a state machine over the rally. At the serve: bind server and receiver from score state and court half; bind partners by elimination; write four binding events to the registry. During the rally: propagate with ReID and motion, confidence decaying. During the stacking window — the 1–2 seconds post-serve when a stacked team migrates to its preferred sides — identity claims carry lower confidence by design; the book renders this as a confidence timeline, not a boolean. At the next serve: re-pin. The net line is an identity barrier throughout: no legal play swaps a player across it, so any apparent crossing is a tracking error to repair, not an identity event to accept.

Rugby league has its own version of rules-as-identity: the 1–17 numbering is a position prior. The 9 sits at the play-the-ball, the 1 sweeps deep, the 6 and 7 run the halves, 13 locks the scrum, 14–17 are the bench. Number ↔ position ↔ spatial zone triangulates: a jersey read of "9" should sit at the ruck; a track in the fullback zone claiming "9" is a read error or an interchange event, and the system should say which. The rule for all rule-priors (restated from chapter 9): soft costs in the assignment, with explicit violation channels — interchanges, mid-game position swaps, stacking variants. Hard rules manufacture confident silent errors, the worst identity failure class.

Pickleball re-identification across a match: four enrollment cards, overhead doubles court, match timeline with serve pins and dashed propagation segments, stacking window highlighted. Recreate: white background, black linework, burnt-orange accents.
Figure 13.6: Rules As Identity Evidence. Enrolled once, deduced at every serve, propagated through the rally, re-deduced at the next. For four players on a fixed court, arithmetic beats recognition.

13.6 Roster Fusion (The Pro Pattern)

The pro-sport multiplier: identity stops being open-world recognition and becomes a constrained assignment problem over a known, small set. With a roster, the assignment space caps at tracks × active players, solvable by Hungarian matching with edge weights from jersey reads, position zones, and ReID scores — before any open-world recognition runs at all.

Three fusions do the work. Roster-constrained jersey decoding: the roster turns OCR from open-set text recognition into classification over at most 17 valid numbers per team. A wobbly read of "1?" becomes "13" if 13 is on the field and 14/19 are benched — this single constraint is worth more than any recognizer upgrade, and it is free. Lineup state as a temporal index: interchange churn, sin-bins, and late scratches change the active set minute by minute; roster priors must be time-indexed or they confidently assign identities to players who are off the field. Wearable joins: Catapult GPS/LPS vests (the NRL-standard layer [verify current deployment terms]) and Kinexon-class UWB systems solve identity by construction — the device is registered to the athlete, so the roster join is the identity. Optical tracks cross-registered to wearable streams get names for free; the hard part is timestamp and clock sync, not recognition. The Sportlogiq / Second Spectrum-style fusion of optical tracking with official lineup data is the commercial form of this pattern [INFERENCE — proprietary internals, no citable specifics].

The amateur translation is the chapter-5 one-spine move: a roster CSV as the identity universe, position priors, visual evidence in fusion order. Even a social pickleball session has a degenerate roster — the four names typed in at enrollment — which is exactly what makes §13.5's deductions closed-world and exact. For rugby, the #7 sitting in the middle of the line is a spatial prior that resolves ruck ambiguity the same way the serve-side rule resolves the NVZ crowd: the sport's structure is free identity evidence everywhere you look.

13.7 The Face Line

Faces are the strongest identifier and the one with the sharpest legal boundary. Facial templates used for unique identification are biometric data — GDPR Article 9 special category (processing prohibited by default absent narrow exceptions like explicit consent, with DPIA obligations) and sensitive information under the Australian Privacy Act with the higher consent bar [verify post-2024-reform wording before print]. The consent problem is structural in sport: consent must be freely given, but a club's power over its athletes — and a coach's power over a social player who just wants a rally report — undermines voluntariness. A teammate cannot meaningfully refuse the club's face-enrollment. For amateurs there is no consent infrastructure at all: the person filming a social doubles game has no basis to biometrically enroll three other people.

The technical case is weaker than vendors admit, and pickleball shows both sides of it. The same sport appears in two camera regimes: court-side fixed cameras (this book's home case) give near-frontal, well-lit faces at enrollment distance — the one regime where face recognition would technically work; and broadcast (pro pickleball tours now have it) gives profile views, motion blur, visors, and 40-metre subject distances — the regime where it degrades to gambling. Rugby broadcast is worse: heads down in the ruck, headgear, mud, and a scrum cap on exactly the player you most need to separate. Meanwhile the error profile is unequal: NIST FRVT demographic-differentials studies document false-positive rates varying by factors of 10–100× across demographic groups in many algorithms (source-backed) — an identity system whose error rate depends on the player's face is a liability with a leaderboard. And a jersey number can be re-issued; a leaked face template is irrevocable.

The book's rule: no face recognition in amateur footage. Period. Every identity need in this book has a non-biometric path — §13.3 to §13.6, plus enrolled appearance galleries with informed session consent, deletable on request. Faces are for broadcast graphics operators identifying public figures under league agreements; the pro-team roster+jersey+wearable stack is more accurate anyway. The pro case is a consent-managed infrastructure decision, not a CV choice (C31 rights discipline).

13.8 Evaluation

Measure identity, not tracking. The metric stack, in order of what it catches:

  • ID-switch rate (IDSW per 1,000 frames): raw identity-break frequency — the E09 14→2 story. A universal smoke signal; never sufficient alone, because it misses the silent failure.
  • IDF1 / ClusteredIDF1: track fragmentation against ground-truth identities; the clustered variant (chapter 9's construction) scores per team/side/role, which is the only honest fragmentation metric for same-kit sport.
  • GS-HOTA (IdSim): localization × attribute-correct identification — role, team, and jersey must all match for identity similarity to count (source-backed, GSR). The end-to-end metric where a minimap ground truth exists.
  • ReID retrieval mAP / CMC Rank-k: can the right identity be retrieved from the gallery — the honest embedding benchmark for validating enrollment before deployment.
  • Jersey tracklet accuracy with abstention: one number per tracklet, "-1" counted correct only when truly illegible (the sn-jersey protocol). Report legibility-gate precision separately.
  • Identity-mixing AUC (book construction, proposed): for each predicted track, the fraction of frames belonging to its modal ground-truth identity, swept over the purity threshold to an AUC. This catches the track that never "switches" but is 60/40 two people — the failure IDSW cannot see.

Two protocol rules bind them all. Resample by clip, not frame (chapter 20): a jersey-vote accuracy computed over 900 consecutive frames is n = 1 rally, not n = 900. And publish binding coverage — the percentage of track-frames with a declarative binding versus propagation-only — next to every accuracy number: a 95%-accurate system over 20% of frames is worse than a 90%-accurate system over 90% of frames for typing stats downstream.

13.8b Failure Modes

Failure Mechanism Mitigation
Occlusion piles (ruck, tackle, NVZ crossing) Crop contamination poisons tracklet votes; tracks merge ReID-gated crop hygiene (outlier removal before the vote); net-line and position priors
Motion blur Per-frame digit reads wrong with high confidence; embeddings smear Legibility gate before OCR; vote over the tracklet, never the frame
Glare / lighting shifts Specular patches erase digits; shadow bands drift embeddings and straddle k-means centroids Embedding clustering over raw color; EMA gallery with updates frozen under ambiguity
Identical kits Appearance space collapses to one blob; clustering and ReID cannot separate Serve-side pinning; enrollment gallery; declare unclusterable rather than force k=2
Fabric fold / mud Digits deform beyond the recognizer's training distribution Pose-guided RoI; abstain ("-1") as a first-class output; never force a read
Broadcast cuts / camera changes Track resets faster than votes accumulate (E08: 295 IDs) GMC (E14); cut detection as a registry boundary; roster-constrained re-assignment after cuts
Roster churn Active-set priors confidently wrong during interchange Time-indexed lineup state; explicit violation channel; soft priors only
Gallery drift-capture EMA gallery locks onto the wrong player after unresolved occlusion Freeze updates under ambiguity; re-pin at the next binding event
Confident silent wrong identity (the compound failure) Hard priors plus no abstention channel The design rule: every claim carries provenance and confidence; hard rules prohibited
Identity failure mode gallery: six cards showing rugby ruck occlusion, motion blur, jersey glare, identical kits, fabric fold, and broadcast camera cut. Recreate: white background, black linework, burnt-orange labels.
Figure 13.7: The Failure Gallery. Every identity module has a characteristic way to break; the registry's job is to degrade confidence, not to keep guessing at full confidence.

13.9 Identity by Audience

Audience Available Evidence Identity Stack
Amateur pickleball (4 players) 4 known people, fixed camera, serve rules, homography Serve-side deterministic pinning + 10-second enrollment gallery + ReID propagation. No faces, no OCR, no forced k=2.
Semi-pro / club Known small roster, kits (possibly clashing) Per-session gallery enrollment + roster-constrained assignment; jersey OCR only if numbered kits.
Club rugby (13 + 13) Numbered kits 1–17, roster, position structure Jersey OCR (localized) + legibility gate + tracklet majority vote + time-indexed roster + position priors.
Pro team The above + wearables + staff to verify Wearable join as ground truth → the club stack → human verification loop on residuals (Catapult/Kinexon, chapter 30) in the one-spine.
Minors / schools Nothing consentable No identity. Role-level analysis only ("near-side server"), anonymized aggregates.

13.9b The 10 Use Cases: Applied Identity Framework

The use cases below are the applied bridge from the fusion stack above to the two sports. They follow three categories: Declarative Reads (UC 01–03) — the binding events that create identity; Propagation & Retention (UC 04–07) — carrying identity through everything that tries to break it; and The Registry & Its Limits (UC 08–10) — the data structure downstream chapters actually consume, and the honest edge where identity stops. Each case carries its evidence label: measured (book experiment), source-backed (paper), or [verify] (practitioner model, not yet established). The tool stack they draw on, with the sport each serves:

Tool Identity Role License / M4 Sport Use
EasyOCR / PP-OCRv5 Jersey digit reads on torso crops (§13.3) Apache-2.0; MPS ✅ / CPU ⚠️ Rugby league back numbers; no pickleball role
PARSeq STR Per-sport fine-tuned jersey recognizer Apache-2.0; train ⛔ GPU Rugby league (the Koshkina recipe, clean-license rebuild)
OSNet-512 (torchreid) Appearance embeddings, enrollment galleries MIT; MPS ✅ Pickleball propagation; rugby propagation between reads
PRTreID Multi-task identity + team + role embeddings [verify repo license]; ⚠️/⛔ Rugby league GSR-style fusion; pickleball overkill
SigLIP → UMAP → KMeans Team clustering prior MIT (roboflow/sports); ✅ Rugby league distinct kits; abandoned on all-black pickleball
Hungarian assignment (scipy) Roster-constrained track-to-name matching BSD; ✅ Both — 4-name pickleball roster, 17-man rugby active set

Category I: Declarative Reads — Where Identity Comes From (UC 01–03)

UC 01 — Jersey Number OCR Under Broadcast Hostility

Thirteen same-kit athletes per side make the back number the only durable name tag on a rugby league broadcast — and it is legible in perhaps 5% of frames (source-backed, the sn-jersey design assumption). The pipeline: person crop (chapter 8) → pose-guided torso RoI (E03's stack, 82 fps on M4) → legibility gate → STR read → digit-regex filter → tracklet majority vote: n* = mode({rt : legible(rt)}), abstaining to "-1" when no read clears the gate. The roster then constrains n* to the active set — a wobbly "1?" becomes "13" only if 13 is on the field. The C13-lab measured the bottleneck: 6/6 nrl-001 frames produced digit reads, but scoreboards and hoardings pollute them — localization, not recognition, is the hard module. The GSR ablation prices it: adding the jersey attribute drops identity HOTA from 37.03 to 25.65, more than calibration, role, and team combined (source-backed, arXiv:2404.11335); the sn-jersey leaderboard tops out at 92.85% tracklet accuracy against 3.93% random. Payoff: this is the only declarative identity read that survives a camera cut — every rugby stat line in chapters 21–23 types off it.

Rugby league jersey number OCR pipeline: rear view of player wearing number 9, magnified torso crop with digit boxes, legibility gate, tracklet majority vote tally.
Figure 13.8: UC 01 — Jersey OCR Under Hostility. Crop the torso, gate legibility, vote across the tracklet, constrain to the active roster. Rugby league: back numbers 1–17 with close-ups after every try; the ruck, mud, and folds supply the failure gallery. Pickleball: no numbered kits — this case does not transfer; UC 02 and the §13.5 serve rule carry pickleball identity.

UC 02 — ReID Across a Rally

A pickleball rally crosses the court, the net, and occasionally the camera's edge; identity must survive all of it with no numbers to re-read. The mechanism is gallery enrollment plus cosine propagation: enroll each known player once (10 seconds alone in frame) into a gallery G = {g1…g4}, then assign each crop's embedding q to the nearest enrolled identity when cos(q, gi) = q·gi / (‖q‖‖gi‖) exceeds the rejection threshold τ, and propagate that binding through the rally until the next serve re-pins it (§13.5). The measured anchor is the tracker underneath: E09's BoT-SORT at HOTA 0.814 with 2 switches on pb-003, and the EMA-gallery drift-capture failure chapter 9 documented when updates run during unresolved occlusion — the reason gallery updates freeze under ambiguity. Payoff: the amateur's entire declarative layer — four enrollment cards turn anonymous track integers into named rally reports with zero annotation.

Pickleball doubles rally ReID propagation: overhead court, four players with embedding glyphs, cosine arrows to four enrollment cards, dashed propagation path.
Figure 13.9: UC 02 — ReID Across a Rally. Enroll once, match by cosine above threshold, propagate between serve-time re-pins. Pickleball: the home case — four players, four cards, one walk-through each. Rugby league: propagation only, never declaration — 15 near-duplicate bodies per kit collapse the embedding neighborhood (chapter 9's same-kit measurement).

UC 03 — Face Recognition Court-Side (The Forbidden Case)

Court-side fixed cameras — this book's home pickleball regime — are the one setup where face recognition would technically work: near-frontal, well-lit, enrollment-distance faces. The mechanism would be a face-embedding gallery matched exactly like UC 02. The math that matters is not the similarity function but the error and legal profile: NIST FRVT demographic-differentials studies document false-positive rates varying 10–100× across demographic groups (source-backed, NIST FRVT), and facial templates used for unique identification are GDPR Article 9 special-category biometrics and sensitive information under the Australian Privacy Act [verify post-2024-reform wording before print]. Consent is structurally broken in sport: a teammate cannot meaningfully refuse the club's enrollment, and a social filmer has no basis to biometrically enroll three friends. The book's verdict is UC-shaped precisely because the temptation is real: the rule stands — no face recognition in amateur footage; every identity need here has a non-biometric path (§13.3–§13.6).

Face recognition consent boundary at a pickleball court: sideline camera, crossed-out biometric template, consent shield, green zone of allowed signals separated from red zone.
Figure 13.10: UC 03 — The Face Line. Technically feasible court-side; legally and statistically prohibited for this audience. Pickleball: enrollment galleries of body appearance with informed session consent, deletable on request. Rugby league: the roster + jersey + wearable stack is more accurate anyway — faces are for broadcast graphics operators under league agreements, not coaching tools.

Category II: Propagation & Retention — Surviving the Match (UC 04–07)

UC 04 — Kit-Change and Substitution Handling

Rugby league rotates its 14–17 bench through interchange churn, sin-bins, and blood bins — the active set of 13 changes minute by minute — and social footage adds half-time kit changes and tournament re-kits. Every one of these is a registry boundary: propagation stops, the interval closes, and a new unbound interval opens until the next binding event. The mechanism is a time-indexed lineup state L(t) — who is actually on the field at time t — joined against incoming tracks: a substitution detected at the sideline (or a halftime timestamp) closes the departed player's interval, and the replacement is bound by roster-constrained jersey read or position prior, never by appearance continuity across the boundary. The failure this prevents is the confident wrong name: a roster prior applied to a player who left the field two minutes ago. Payoff: interchange-accurate stat typing — running metres and tackle counts land on the player who was actually on, which is the difference between a workload report and fiction (feeds chapters 21 and 30).

Kit change identity handling: rugby interchange board, player leaving in stained kit and fresh player entering, registry interval closing and reopening, clock icon.
Figure 13.11: UC 04 — Kit Change as Registry Boundary. Close the interval, open unbound, re-bind from the time-indexed roster. Rugby league: interchange, sin-bin, blood bin — L(t) or the priors lie. Pickleball: side-switches at 11 and between-game re-kits; the serve-side rule re-pins identity for free after every boundary.

UC 05 — Same-Kit Disambiguation (13 vs 13)

The hardest identity configuration in either sport: twenty-six athletes, two kits, and an appearance space that has already collapsed to two blobs — chapter 9 measured the same-kit embedding collapse, and §13.4's clustering cannot go below team level. The mechanism is the full fusion stack as an assignment problem: team clustering halves the search space, jersey reads declare where legible, position priors (1 fullback deep, 9 at the play-the-ball, 6/7 in the halves) sanity-check, and Hungarian matching over tracks × active roster resolves the rest with edge weights w = α·(jersey read) + β·(position zone) + γ·(ReID score), all soft costs with violation channels for interchanges and position swaps. The triangulation rule does the honest work: a track in the fullback zone claiming "9" is a read error or an interchange event, and the system must say which. Payoff: this is the pro-club identity core — the assignment frame is what turns six noisy signals into one defensible name per track (the GSR architecture, source-backed).

Same-kit disambiguation in rugby league: two teams in identical dark jerseys, merged embedding cluster panel, jersey reads and position zones separating players, assignment matrix.
Figure 13.12: UC 05 — Same-Kit Disambiguation. Clustering halves the space; reads, zones, and the roster solve the rest. Rugby league: the 13v13 match reality — number ↔ position ↔ zone triangulation. Pickleball: the all-black social game — the same collapse at n=4, solved by the serve rule and enrollment instead of reads and zones.

UC 06 — Occlusion Identity Retention (The Net, The Ruck)

Identity evidence dies in piles: the rugby ruck buries numbers under bodies, the tackle deforms the kit, and the pickleball NVZ crossing merges tracks exactly when the rally is hottest. The retention mechanism is disciplined decay plus re-binding: when occlusion is detected, freeze gallery updates (the drift-capture guard), freeze crop contributions to the tracklet vote (ReID-gated crop hygiene — the Koshkina outlier step, source-backed), and let confidence decay as c(t) = c0·e−λΔt with λ scaled by occlusion density and kit ambiguity. When the pile separates, re-bind from the first declarative read — a jersey flash, a position prior, the serve-side rule — rather than trusting the embedding bridge across the gap. The prohibited move is updating anything during the pile: every frame inside an occlusion is contamination, not evidence. Payoff: this is where identity-mixing is born and where it is prevented — retention discipline is the difference between a 60/40 polluted track and an honest confidence dip (§13.8's identity-mixing AUC measures exactly this).

Occlusion identity retention: rugby ruck pile with overlapping players, suspended identity tokens with decaying confidence bars, pickleball net inset, re-binding arrow.
Figure 13.13: UC 06 — Retention Through the Pile. Freeze updates, decay confidence, re-bind at separation. Rugby league: the ruck and the tackle — numbers buried, votes frozen. Pickleball: the net-line is an identity barrier — no legal play crosses it, so apparent crossings are tracking errors to repair, not identity events to accept.

UC 07 — Camera-View Identity (The Chapter-14 Feed)

Broadcast coverage cuts between the wide tactical camera, the close-up, and the replay faster than any tracklet vote can accumulate — E08's 295 spawned track IDs for ~30 humans on nrl-001 are mostly the director's fault, and E14's global-motion compensation only recovered nine. Each cut is a registry boundary: the old interval closes and the re-identification problem restarts. The cross-view mechanism fuses three weak signals into one assignment: appearance embeddings (weak across viewpoint and zoom), homography-mapped position on the shared minimap (chapter 14's multi-view geometry — a track leaving the left edge of frame A must enter frame B at a consistent pitch location), and the roster constraint (the re-entering track must map to someone actually on the field). No single signal is trusted across a cut; the assignment is. Payoff: this is what makes broadcast footage analyzable at all — without cross-view identity, chapter 14's minimap is a crowd of anonymous dots and every per-player broadcast stat is unreachable.

Multi-camera identity handoff: two camera views of a court, track leaving one frame and entering the other, embedding match bridge, overhead minimap with camera cones.
Figure 13.14: UC 07 — Identity Across the Cut. Embedding + minimap position + roster, assigned jointly. Rugby league: the broadcast reality — wide to close-up to replay, 295 track fragments converging to ~30 names. Pickleball: multi-camera club rigs and phone-plus-fixed-camera setups; the homography bridge is cheap on a fixed court (E05/E11-grade, 3.82–4.65 cm RMSE).

Category III: The Registry & Its Limits (UC 08–10)

UC 08 — The Identity Registry (The One-Spine Feed)

Every downstream chapter consumes identity through one structure, and this use case is it: the registry mapping (track_id, t_start, t_end) → identity, confidence, provenance. The mechanism is the write discipline from §13.2b made concrete — binding events (jersey reads, serve-side deductions, enrollment matches) write declarative intervals with provenance tags; propagation extends them probabilistically; boundaries (cuts, substitutions, kit changes) close them. The deliverable property is the join: chapter 5's one-spine stores nothing but intervals, so "player 9's running metres" is computed as a join against named intervals, never against bare track integers — which is what makes identity-mixing detectable instead of silent. A registry row reads like a receipt: "track 147 → player 9, confidence 0.91, bound by jersey read at 54:12, propagated 210 frames." Payoff: the single point where tracking becomes analytics — chapters 21–23 and 28 all query this table, and its provenance column is what makes their numbers auditable.

Identity registry table: track timeline with intervals mapped to names, confidence bars, provenance tags, binding discs and dashed propagation, downstream stats join.
Figure 13.15: UC 08 — The Registry. Intervals, confidence, provenance — the one-spine's identity table. Pickleball: serve pins as binding events, ReID propagation between them, per-player rally reports off the join. Rugby league: jersey reads and lineup state as bindings, propagation through rucks and cuts, workload stats typed by interval.

UC 09 — Confidence Calibration (The p-Gate)

A registry that emits bare names is a liability; the product is calibrated confidence with an abstention channel. The mechanism: every binding score — jersey-vote share, cosine margin, assignment posterior — is temperature-scaled against held-out clips (chapter 20's clip-level resampling, since 900 consecutive frames are n = 1 rally), and every propagated claim carries c(t) from UC 06's decay. The p-gate is the operating rule: claims below threshold τ are emitted as "unknown," never rounded up to the nearest name; reliability diagrams verify that a 0.9 claim is right 90% of the time per clip. The companion metric is binding coverage — the percentage of track-frames with a declarative binding versus propagation-only — published next to every accuracy number, because a 95%-accurate system over 20% of frames types worse stats than a 90%-accurate system over 90%. Payoff: calibration is what lets a coach act on the number — the p-gate converts identity from a guess into a decision-grade input with a stated error rate.

Confidence calibration gate: probability axis with abstain zone below threshold and accept zone above, reliability diagram, pickleball and rugby player score markers.
Figure 13.16: UC 09 — The p-Gate. Temperature-scaled scores, abstain below τ, reliability checked per clip. Pickleball: stacking-window claims auto-discounted — the confidence timeline dips by design. Rugby league: post-cut and post-interchange intervals start unbound and earn confidence only from new declarative reads.

UC 10 — The Transient Identity Failure (The Honest Limit)

Sometimes identity is simply unknowable: a broadcast cut, a kit change, and two minutes off-frame stack into a gap no signal bridges — the registry's correct output is "unknown," and the honest system is the one that says so. The mechanism is the failure protocol: when confidence collapses through the floor (decayed below τ with no binding event in reach), close the interval, mark the gap, and wait — never force a read, never let a hard prior fill the silence, never let the EMA gallery drift-capture a convenient neighbor. The compound failure this use case exists to prohibit is the confident wrong name: a hard prior plus no abstention channel manufacturing certainty the evidence does not contain — the one failure class §13.8b bans by construction. The transient framing is the comfort: identity in sport is episodic by nature, and a gap is a normal state with a recovery path (the next serve, the next close-up, the next legible number), not an error to paper over. Payoff: trust — the analyst who can see the gaps believes the fills, and the system that admits "unknown" for 8% of frames keeps its other 92% actionable.

Transient identity failure: timeline where confidence collapses after a broadcast cut and kit change, unknown label card, abstain stamp, recovery at next binding event.
Figure 13.17: UC 10 — The Honest Limit. Close the interval, mark the gap, wait for the next binding event. Rugby league: cut + interchange + off-frame minutes = a declared unknown, recovered at the next close-up read. Pickleball: a player fetching a ball re-enters unbound and is re-pinned at the next serve — seconds of "unknown," zero wrong names.

13.9c What This Adds to the Pipeline

The ten use cases are the identity layer applied across the book's five verbs. UC 01–03 feed watching and tagging: the declarative reads that let chapter 17's event attribution and chapter 19's eventing name an actor instead of a track integer. UC 04–07 protect modelling: retention and cross-view identity are what make chapter 14's multi-view minimap and chapter 22's EPV surfaces player-typed rather than crowd-typed. UC 08–10 serve interpreting and acting: the registry is chapter 5's one-spine identity table, the p-gate is chapter 20's calibration discipline applied to names, and the chapter 28 cockpit displays exactly what the registry vouches for. Chapter 30's wearables enter as identity ground truth (the UC 04 lineup join), and chapter 31's rights discipline owns the UC 03 wall. The honest labels mark which numbers are measured in this book's lab (C13-lab, E08/E09/E14), which are source-backed (GSR, sn-jersey, NIST FRVT), and which remain [verify] — and UC 10 is the standing reminder that the system's most valuable output is sometimes the word "unknown."

13.10 What I Would Measure Next

  • Jersey-region OCR: person-detector → pose-guided torso crop → digits on 50 hand-labeled nrl-001 tracklets under the sn-jersey protocol ("-1" for never-visible): tracklet accuracy, legibility-gate precision, and the mud/fold/ruck failure gallery — the first public rugby jersey-OCR baseline.
  • Tracklet majority vote on a 10-second rugby clip: does the mode stabilize the number, and how many frames does convergence take after a broadcast cut?
  • Deterministic doubles identity on pb-003: serve-side pinning over E05/E11 homography feet + score state, BoT-SORT propagation, re-pin every serve — identity confidence timeline per player plus ClusteredIDF1 against hand labels; the worked proof that rules beat recognition for 4-person sport.
  • Per-session gallery for pickleball: enroll 4 players once, measure ReID retrieval mAP/Rank-1 and identity-mixing AUC against the rule-pinned output — the amateur fallback's accuracy floor.
  • Team-clustering bake-off: raw-color k-means vs SigLIP→UMAP→KMeans on nrl-001 (expect both work) and pb-003 (expect collapse) — per-frame team-label flicker and tracklet-vote stability, making §13.4's honesty paragraph empirical.

13.11 Sources

  • SoccerNet GSR (arXiv:2404.11335) and sn-jersey (github.com/SoccerNet/sn-jersey) — the open jersey-OCR reference, GS-HOTA ablation, and the 92.85% tracklet benchmark; TrackLab framework (MIT). Dataset license: SoccerNet research terms [verify redistribution].
  • Koshkina & Elder, "A General Framework for Jersey Number Recognition in Sports Video," CVPRW 2024 (CC BY-NC 3.0 — learn from it, don't ship it); PARSeq STR (Apache-2.0) as the clean-license recognizer; Liu et al., pose-guided R-CNN, CVPRW 2019.
  • EasyOCR (Apache-2.0) — the C13 lab reader; PaddleOCR / PP-OCRv5 (Apache-2.0, CPU-only on M4) as the alternative; roboflow/sports (MIT) — the SigLIP→UMAP→KMeans team-clustering pattern.
  • Ethics lane: ICO biometric recognition guidance (GDPR Art. 9); Australian Privacy Act sensitive-information class [verify post-2024 wording]; NIST FRVT demographic differentials (10–100×); Privacy International, "Sports and Surveillance."
  • Book-internal: E01, E02, E03, E05, E08, E09, E11, E14 (verified lab results); C05 (one-spine), C08 (detection), C09 (tracking/ReID depth), C20 (statistics), C30 (wearables), C31 (rights).
  • Lab: lab/w3_lab_identity.pyexperiments/c13-identity/outputs/metrics.json (6/6 frames read digits; localization is the bottleneck — measured).

Next Chapter

Chapter 13 — Identity: Who Is Who

Track IDs are not people: jersey OCR, ReID embeddings, team clustering, roster priors, and the consent line

Continue Reading
AS '26

Agentic Sport Analytics

A practitioner's field guide to automated sport analytics: watching, tagging, modelling, interpreting, and acting with AI, LLMs, computer vision, and agent harnesses. Measured on pickleball and Australian rugby league. By Mehran Mozaffari. First Edition, August 2026.

Front Matter

Preface

Front Matter

Copyright & License

Watching

Chapter 01 — Build the Lab, Not the Manuscript

Watching

Chapter 01 — Why This Book Exists: The Five Verbs of Sport Analytics

Watching

Chapter 02 — The Evidence Contract & Data Provenance

Watching

Chapter 02 — The Evidence Contract & Data Provenance

Watching

Chapter 03 — Calibrating the World: Homography & Court Geometry

Watching

Chapter 03 — Sport Rules as Formal Systems

Watching

Chapter 04 — Finding & Tracking the Actors: From ByteTrack to Meta SAM 2/3

Watching

Chapter 04 — Capture: Cameras, Lenses, Shutter, Placement

Watching

Chapter 05 — The Body in Motion: 2D Keypoints to Meta SAM 3D Body

Watching

Chapter 05 — Data Engineering for Sport Video

Watching

Chapter 06 — Smashing the Ball Wall: Spatio-Temporal Trajectory Recovery & SAM 2/3 Equipment Segmentation

Watching

Chapter 06 — Calibration I: Homography, Intrinsics, Distortion

Watching

Chapter 07 — The Structured Representation: PBN & State Machines

Watching

Chapter 07 — Calibration II: Broadcast Dynamics, GMC, and Per-Frame H_t

Tagging

Chapter 08 — Reading Space & Pressure: Geometric Deep Learning

Tagging

Chapter 08 — Detection: YOLO, RF-DETR, and the AGPL Decision

Tagging

Chapter 09 — Generative Replay & Counterfactual Simulation

Tagging

Chapter 09 — Tracking & Identity: Metrics, ReID, and Role Priors

Tagging

Chapter 10 — Where Vision-Language Models Help, and Where They Lie

Tagging

Chapter 10 — Segmentation & Foundation Models: SAM 2/3, DINOv3

Tagging

Chapter 11 — Building the Live Coaching Cockpit on Apple Silicon

Tagging

Chapter 11 — The Body in Motion: 2D Keypoints to 3D Biomechanics

Tagging

Chapter 12 — Complex Motion & Field Sport Scaling

Tagging

Chapter 12 — Smashing the Ball Wall: Spatio-Temporal Trajectory Recovery

Tagging

Chapter 13 — Evaluation, Rights, and the Next 10 Runs

Tagging

Chapter 13 — Identity: Who Is Who

Tagging

Chapter 14 — Multi-Camera Geometry, Line Calls, 3D Reconstruction

Tagging

Chapter 15 — Audio & Multimodal Cues: The Free Sensor

Tagging

Chapter 16 — Video Understanding: Action Recognition, Spatio-Temporal

Tagging

Chapter 17 — Event Data & the Common Representation

Modelling

Chapter 18 — Annotation: The Ground-Truth Workflow

Modelling

Chapter 19 — Automatic Eventing: State Machines, Confidence, Review Queues

Modelling

Chapter 20 — Statistics for Sport Practitioners

Modelling

Chapter 21 — Rating Systems: DUPR, ELO, Glicko, and Skill

Modelling

Chapter 22 — Expected Value: xG, VAEP, EPV, and Their Sport Transplants

Modelling

Chapter 23 — Tactical ML: Graphs, Equivariance, and Honest Forecasting

Modelling

Chapter 24 — Simulation & Counterfactuals: The Honest Rebuild

Interpreting

Chapter 25 — Where Vision-Language Models Help, and Where They Lie

Interpreting

Chapter 26 — From Numbers to Narrative: Reports, Scouting, Coach UX

Interpreting

Chapter 27 — The Agent Harness for Sport Analytics

Interpreting

Chapter 28 — The Live Coaching Cockpit: Real-Time Systems, Honestly Measured

Acting

Chapter 29 — Practice Design & Interventions: The Acting Loop

Acting

Chapter 30 — Sensors & Hardware: Wearables, Smart Courts, the Fusion Spine

Acting

Chapter 31 — Deployment, Licensing, Rights & Ethics

Acting

Chapter 32 — The Laboratory: Reproducing the Book's Claims

Acting

Chapter 33 — The Frontier: What's Changing in 2025-2026

Acting

Chapter 34 — The Book as a System: How to Use It (Human + Agent)

©2026 Mehran Mozaffari. Free for personal/noncommercial use (CC BY-NC-ND 4.0); commercial license required for business use.