BeaconBench
Why
Two ground terminals on a laser communication link have to find and hold onto each other before any data can move. That starts with a camera finding a small bright spot on a screen and never losing it, even while the air itself is distorting the view, the platform is shaking, and the sensor is noisy. I built a testbed for that problem against a real published set of performance targets, not a benchmark I made up myself.
How
A simulated camera scene with a moving beacon, checked against real turbulence theory rather than an invented noise model, run through eleven disturbance scenarios from the published specification: vibration, salt-and-pepper noise, moving platforms, dropouts. A classical tracker is the baseline; a learned scorer was tried alongside it and kept only where it actually helped. Every run is scored against the same five numeric targets the specification publishes, and a separate adversarial review was run against the whole thing on purpose, to find what the headline numbers do not show.
What came out
9 of 12 disturbance scenarios meet all five published targets. Re-acquisition after a forced dropout lands at 0.80 s against a 1 s target, and throughput runs 22 to 41 FPS against a 20 FPS minimum. Full-screen search for a beacon that could be anywhere on a 2000x2000 px field does not meet its 2-second target at any setting measured, and the single hardest scenario, every disturbance at once, fails outright. Both are published rather than quietly left out.
What broke
A separate adversarial review of the whole project, run on purpose rather than skipped, found failures the summary numbers above do not show: motion tracking fails at default settings in clear air with nothing else going wrong, a full-resolution supplied video runs at about a quarter of the required frame rate, the tracker can lock onto noise and hold there, and one disturbance test does not actually match what the specification asks for. Every one of those is now a numbered task instead of a quiet gap.
Mind the gap
Built for a national engineering challenge with lane partners; this station says nothing about the team, matching every other station on the site. The repository is private and the competition stays unnamed here until the submission closes, then both go up.