Adapters for your stack
Connect an HTTP endpoint, OpenAI Responses or Chat Completions, Anthropic Messages, or a LangChain Runnable. Extend the adapter interface for your own app.
Choose an adapter → Instrument your agent
Implement a small agent target for your own tool loop, or connect through the optional controlled MCP harness over in-memory, stdio, or loopback HTTP transports.
Instrument agent trials → Portable evidence
Export versioned evidence bundles, regrade saved observations offline, compare a run with a recorded baseline, and review results as JUnit or standalone HTML.
Share and replay evidence → Bounded agent campaigns
Run many isolated trials with shared call, turn, and time budgets, bounded concurrency, and cancellation cleanup. Every planned case keeps a result row.
Bound agent campaigns → Parallel async probes
Use async adapters and the probe runner to test a corpus concurrently, with a limit on simultaneous requests and results for each prompt.
Run probes in parallel → Optional model judging
Add semantic checks through DeepEval, or combine multiple judges with majority, unanimous, or any-pass consensus. Your model provider may charge for calls.
Configure LLM judges → Deterministic grading does not need a judge model. Your chatbot's or agent's own API usage can still incur costs. Passing a fixed test corpus is a regression signal, not a security certification. The versioned coverage matrix lists which risk categories are tested and which are not.