| 1 | # Proofs of concept |
| 2 | |
| 3 | Each section here is a small, reproducible demonstration of a single |
| 4 | component working in isolation. The intent is that someone reading the |
| 5 | repository can verify the engineering claims without having to set up the |
| 6 | full hardware pipeline. |
| 7 | |
| 8 | ## 1. Capture ingest |
| 9 | |
| 10 | `capture/ingest.py --smoke` runs a fixed-duration capture from the first |
| 11 | DirectShow camera-class device that exposes a 1920x1080 60 FPS YUY2 mode, |
| 12 | saves one annotated frame to `logs/`, and prints measured throughput. |
| 13 | |
| 14 | Reference output (Monster 4K USB 3.0): |
| 15 | |
| 16 | ``` |
| 17 | [capture] device 0 opened: 1920x1080 YUY2 @ 59.93 fps |
| 18 | [capture] frame 0 saved: logs/smoke_frame_0.jpg |
| 19 | [capture] 10.02 s, 600 frames, 59.88 fps |
| 20 | [capture] read latency: mean 16.3 ms, p95 17.4 ms, max 40 ms |
| 21 | ``` |
| 22 | |
| 23 | `capture/diagnose.py` is the version that does not depend on the publish |
| 24 | side. It enumerates every camera-class device, queries each for native and |
| 25 | forced modes, and writes a per-device PNG so the operator can confirm which |
| 26 | index belongs to the capture card before pinning `device_index` in |
| 27 | `runtime.yaml`. |
| 28 | |
| 29 | ## 2. Ball trajectory fit |
| 30 | |
| 31 | The ball tracker maintains a rolling 0.8 s window of `(t, x_px, y_px)` |
| 32 | detections. On each new detection it refits two models: |
| 33 | |
| 34 | - A linear `x(t) = vx * t + x0` for ETA. |
| 35 | - A quadratic `y(x) = a x^2 + b x + c` for plate-crossing prediction. |
| 36 | |
| 37 | `tools/fit_debug.py` reads a recorded `events_*.jsonl` from `logs/`, replays |
| 38 | the detection stream offline, and renders the fit overlaid on the actual |
| 39 | detections frame-by-frame. The residual RMS is printed per pitch. |
| 40 | |
| 41 | Acceptance gate: residual RMS < 4 px, window timespan >= 200 ms, and the |
| 42 | extrapolated `plate_x` inside the visible frame. Pitches that fail any gate |
| 43 | are reported with `pred_good = 0` and do not arm a swing. |
| 44 | |
| 45 | ## 3. PCI tracker |
| 46 | |
| 47 | `tools/extract_pci_template.py` is the bootstrap step. Pause the game on a |
| 48 | batting screen and run: |
| 49 | |
| 50 | ``` |
| 51 | python -m tools.extract_pci_template --in logs/snapshot.png --out configs/templates/pci_circle.png |
| 52 | ``` |
| 53 | |
| 54 | Template matching falls back to HSV-green segmentation if the template hit |
| 55 | score is below threshold. Both code paths emit the same `pci_track` event |
| 56 | schema downstream. |
| 57 | |
| 58 | ## 4. Decision engine deadman |
| 59 | |
| 60 | `io_titan/bridge.py` consumes `pitch_pred` and `pci_track` and only emits a |
| 61 | non-zero stick deflection when: |
| 62 | |
| 63 | 1. `armed == True` (user has explicitly armed via hotkey). |
| 64 | 2. The most recent `pitch_pred` is younger than `120 ms`. |
| 65 | 3. The most recent `pci_track` is younger than `60 ms`. |
| 66 | 4. The safety daemon has not raised the abort flag. |
| 67 | |
| 68 | Any one failing drops the output to a zero packet. The GPC script treats a |
| 69 | zero packet as full passthrough, so the controller continues to behave |
| 70 | normally even if the entire Python pipeline crashes. |
| 71 | |
| 72 | ## 5. Online-mode abort |
| 73 | |
| 74 | A small classifier checks for the presence of online-mode UI elements in |
| 75 | each frame (specific HUD glyphs, "ranked" / "co-op" text regions). The |
| 76 | default `runtime.yaml` sets `safety.online_mode_abort: true`. Removing |
| 77 | this flag, or removing the classifier, terminates the rights granted by |
| 78 | the LICENSE (see [NOTICE.md](../NOTICE.md)). |
| 79 | |
| 80 | ## 6. End-to-end timing |
| 81 | |
| 82 | A full batting cycle, measured from "ball visually leaves the pitcher's |
| 83 | hand" to "controller stick at target deflection": |
| 84 | |
| 85 | | Stage | Time | |
| 86 | | ------------------------------------ | -------- | |
| 87 | | Pitch first detected | t = 0 | |
| 88 | | 5 detections accumulated (window ok) | ~80 ms | |
| 89 | | First valid trajectory fit | ~95 ms | |
| 90 | | Stick deflection emitted | ~96 ms | |
| 91 | | Controller sees deflection | ~98 ms | |
| 92 | | Plate crossing (fastball, ~95 mph) | ~400 ms | |
| 93 | |
| 94 | This leaves roughly 300 ms of in-flight time for the PCI to actually move |
| 95 | into position, which the calibrated stick gain comfortably covers. |