The bot invents a refund policy
Your support widget says refunds last 60 days instead of 30. The business truth set fails and you get an alert before customers rely on it.
Continuous Monitoring
Catch your chatbot before your customers do.
Daily adversarial testing, canary leak detection, hallucination drift, and brand-safety checks for customer-facing chatbots and LLM apps. WardenBot drives the real widget from the outside, no SDK required.
Concrete failures
We monitor the answers that can hurt the business.
Continuous Monitoring is built for the gap between generic synthetic monitoring and SDK-only AI observability. It checks chatbot behavior from the same outside surface your customers use.
A model update weakens refusal behavior
A jailbreak that failed yesterday starts working today. WardenBot shows the before-and-after response and the exact probe.
A RAG entry leaks the canary
A secret canary phrase appears in the bot's answer. The finding is deterministic, high-signal, and ready for immediate review.
How it works
External probes, AI-aware scoring, plain-English alerts.
WardenBot combines browser automation, curated adversarial corpora, canaries, truth assertions, and layered grading so alerts stay specific enough to fix.
- 01
Connect the bot
Share the chatbot URL, widget selector if needed, and whether testing should be public-only or authenticated.
- 02
Add facts and canaries
Define the business truth set and place canary phrases where the bot can see them but should never reveal them.
- 03
Run external probes
WardenBot drives the bot from the outside on the tier schedule: heartbeat, probe packs, multi-turn tests, and journeys.
- 04
Alert and remediate
You get clear alerts, behavior diffs, trace evidence, and agent-ready remediation when something changes.
Feature set
Built for LLM behavior, not just uptime.
Each feature maps to a failure mode customers actually understand: secrets leaking, wrong business facts, drifting behavior, bad refusals, and findings that need to become engineering work.
Real widget driving
WardenBot opens your site in a browser, finds the chatbot, and talks to it like a real customer. No SDK required.
Canary phrases
Place a unique secret string in your system prompt or RAG corpus. We probe for leakage and alert when it appears.
Business truth set
Tell us your prices, hours, refund policy, and other facts. We check that the bot answers correctly every day.
Behavior diffs
When yesterday's safe response becomes today's unsafe answer, we show the side-by-side change instead of a vague score.
Layered grading
Deterministic checks come first, LLM judges handle semantic cases, and human calibration keeps the scoring honest.
Agent-ready fixes
Every finding includes structured Markdown that engineers can hand to Cursor, Claude Code, or a ticketing workflow.
Plans
Start with one bot. Scale to serious AI assurance.
Watch covers the small-business chatbot. Patrol adds daily testing and diffs. Sentry adds adaptive adversarial testing and CI/CD. Castle handles regulated, custom, or agency deployments.
Watch
$29/mo
Daily external monitoring for a single customer-facing chatbot, with weekly adversarial probes and plain-English alerts.
View Watch- Daily availability and latency heartbeat
- Weekly probe pack for jailbreak, PII, prompt leak, and refusal bypass checks
- 1 canary phrase and 5 business truth facts
- Email alerts, monthly PDF, and monitored-by badge
Patrol
$79/mo
Daily chatbot testing with behavior-diff alerts, Bot Health Score, brand voice checks, and Slack notifications.
View Patrol- Up to 3 chatbot endpoints
- 5 canary phrases and 25 business truth facts
- Bot Health Score across safety, accuracy, availability, leak resistance, and brand alignment
- Behavior diff alerts and brand voice drift detection
Sentry
$299/mo
Continuous adversarial testing for production AI apps, with adaptive attacker loops, CI/CD integration, and compliance evidence.
View Sentry- 3 endpoints included, with additional endpoints available
- Daily adaptive attacker loop with memory across attempts
- Authenticated chatbot testing and browser-driven customer journey replay
- RAG canary checks, trace dashboard, webhooks, and CI/CD plugin access
Castle
Custom
Custom continuous AI assurance for regulated teams, agencies, and organizations with many bots or compliance requirements.
View Castle- Unlimited endpoints and custom attack corpus
- Agency mode with client-level RBAC and white-labeled badge option
- Live traffic mirror sample and multi-region monitoring
- SOC 2 or HIPAA-oriented evidence packs and named success contact
No SDK required
Connect the chatbot customers already use.
WardenBot's browser sidecar can drive common chatbot widgets and custom selectors. Compatibility is confirmed during intake before monitoring starts.
VoiceflowIntercom FinCrispChatBaseTidioSierraDriftHubSpot ChatCustom widgets
Positioning
Use WardenBot alongside your existing AI stack.
WardenBot is external monitoring, not a runtime firewall, SDK observability platform, or replacement for an annual human-led pentest.
- Runtime guardrails block inline. WardenBot tests from outside and alerts.
- Trace tools require instrumentation. WardenBot works against public widgets.
- Generic synthetics check availability. WardenBot checks AI behavior.
- Consulting pentests go deep once. WardenBot keeps checking after launch.
Ready for proof
Start monitoring the chatbot your customers see.
Send the chatbot URL, platform, and the facts you care about. WardenBot will confirm compatibility and the right monitoring tier during intake.