Open-loop agents write code, run tests, and submit PRs. But they can't see what they built. When a UI regression ships, they don't know until a human notices. We built the infrastructure to close this loop: agents that can browse the product they're building, capture visual sequences, and reason about regressions before a human ever sees them.
Sprites: teaching agents to see motion
LLMs don't understand video natively. But they can reason about sequences of images. We built a format we call 'sprites' — sequences of 16 screenshots taken at regular intervals that capture motion and state transitions. An agent reviewing a UI change can look at a sprite and identify that a dropdown no longer animates, or a loading state persists longer than expected.
When CI modifies the UI, the system automatically captures screenshots and compares against sprite baselines. Differences are surfaced as structured observations — not just pixel diffs, but semantic descriptions of what changed and why it might matter.
The feedback architecture
- Agents launch a headless browser against the preview deployment of their own PR
- Interaction sequences (click, fill, navigate) are replayed automatically
- Screenshots are captured at each interaction step
- Sprites are generated and compared against stored baselines
- Semantic diff summaries are attached to the PR as structured comments
The result is that bugs which would have taken days to surface through manual QA are now found within the CI run that introduced them. Agents can iterate on visual regressions the same way they iterate on type errors.