The problem
An agent that works in a demo fails in production in small, repeated ways. Teams learn about it from users, read traces by hand, and hope a fix does not break another case.
How it works
Find and fix
- Production tracesLangfuse, LangSmith, OpenTelemetry
- grouped into
- Repeated failures
- coding agent proposes
- Fixprompt, tool or code
- replayed on
- Past conversationsin a world rebuilt from traces
Ship
- Pull request with evidence
- a person approves
- Fix ships
- joins
- Protected setlater changes cannot make these worse
The replay rebuilds the apps and data the agent used from its traces, so it never touches real systems.