Free trial
The homepage links to a free start in 5 minutes via the app, but does not state trial length or terms.
Learn moreLangWatch
Trace, test, route and govern every LLM call in your company
Advanced API
What is LangWatch?
LangWatch is an AI simulation generator tool. Agent observability and evaluation platform for AI platform teams: simulated users, red teaming, LLM judges, OnlineTracing via OpenTelemetry, an AI gateway with budgets, and a governance layer with self-hosting and ISO 27001.
LangWatch is an observability and evaluation platform aimed at AI platform teams rather than solo builders. It sits over the LLM calls your company makes, whether those come from agents your engineers wrote, coding assistants like Claude Code and Codex, or providers routed through its gateway. The homepage frames the problem plainly: an agent can reach the same goal by a hundred different paths, and manual testing only covers a handful of them before something breaks in front of a customer. The product is organized in three layers. Simulate spins up simulated users in text and voice that push your agent turn after turn, with scenarios written in plain language (including from inside Claude Code), red teaming runs to probe for jailbreaks and unsafe tool calls, and a judge agent reads the full trace before returning a verdict with its reasoning attached. Evaluate scores outputs and whole conversations with LLM judges, custom code or full workflows, available both from a Jupyter notebook and from a shared UI, plus pairwise comparisons and multimodal scoring for images. Observe covers tracing itself: OpenTelemetry native with GenAI spec support, waterfall and flame graph views, topic clustering, custom metrics graphs and plain language search across large trace volumes. Two adjacent pieces round it out. The AI gateway routes OpenAI, Anthropic, Google, Bedrock, Mistral and others through one endpoint with per-person and per-team budgets, and an agent called Langy turns a product manager's written goal into a test plan, runs the scenarios, scores them and drafts prompt revisions as pull requests. This is a serious tool for teams with real production traffic and compliance questions, not a first experiment. It deploys self-hosted via Docker or Kubernetes, hybrid, or as managed cloud with EU, US, UK and APAC regions, and carries ISO 27001 and GDPR claims. Compared with Langfuse, which one customer explicitly migrated from, LangWatch leans harder on simulation and collaboration.
How do you use LangWatch?
1Open the dashboard and connect a first source
Sign in to the app and choose a path in: instrument your own agent with the Python or TypeScript SDK, or bring coding agents in with one command. Traces from all sources land in the same place.
2Instrument your agent with OpenTelemetry
Follow the OpenTelemetry guide to emit GenAI spec spans. Existing LangChain, LangGraph, CrewAI or Vercel AI SDK setups send traces without rewriting the agent.
3Write your first scenario
Describe the behavior you want to test in plain language, from your editor or the UI. Scenario turns that description into a runnable test with a simulated user.
4Add an evaluator to score the run
Build an eval using an LLM judge, custom code or a workflow, applied to a single output or an entire conversation. The judge reads the whole trace and returns its reasoning.
5Run red teaming before users find the gaps
Point adversarial simulations at your agent to probe for jailbreaks, policy breaks and unsafe tool calls, then fix what surfaces and re-run.6Move scenarios into CI
Wire the same scenarios into your pull request pipeline so every release is tested with no separate setup, catching regressions before merge.
Pros and cons
Pros
- Simulated users in text and voice go beyond static test sets by pushing the agent across multi-turn conversationsAI
- Scenarios can be written in plain language from inside Claude Code, and run locally and in CI with no separate setupAI
- OpenTelemetry native with GenAI spec support means existing traces from LangChain, CrewAI, Vercel AI SDK and others land without a rewriteAI
- The gateway puts every provider behind one endpoint with per-person and per-team budgetsAI
- Deployment flexibility is real: self-hosted on Docker or Kubernetes, hybrid, or managed cloud with EU, US, UK and APAC regionsAI
Cons
- Pricing is not published anywhere on the homepage, so budgeting requires a sales conversationAI
- The surface area (simulations, evals, tracing, gateway, governance) is large and assumes an existing platform team to operate itAI
- Agent frameworks are well covered, but teams on older or in-house stacks may find the integration path less smoothAI
- Simulated users and LLM judges are still models judging models, so verdicts need auditing rather than blind trustAI
- Feature depth suits organizations with compliance reviews; smaller teams may pay for governance controls they never useAI
How much does LangWatch cost?
Support
Demo booking with a solutions engineer and a self-hosting guide are linked from the homepage. A trust center covers certifications and compliance documentation.
Learn moreIntegrations
Python and TypeScript SDKs plus plain OpenTelemetry, with LangChain, LangGraph, CrewAI, Mastra, Pydantic AI, Vercel AI SDK and n8n. Coding agents Claude Code, Codex, Copilot and opencode via one command. Gateway covers OpenAI, Anthropic, Google, Bedrock, Azure, Mistral, Groq, DeepSeek and Vertex AI.
Learn moreFeatures
Simulated text and voice users, scenarios written in plain language, red teaming for jailbreaks and unsafe tool calls, a judge agent with trace reasoning, LLM-as-judge and code evaluators, offline and online evaluation, pairwise and multimodal scoring, OpenTelemetry native tracing, waterfall and flame graph views, topic clustering, custom metric graphs, an AI gateway with per-person and per-team budgets, and governance controls including RBAC, SSO, SCIM and audit logs to SIEM.
Learn moreFrequently asked questions about LangWatch
Does LangWatch offer a free trial?
The homepage links to a free start in 5 minutes via the app, but does not state trial length or terms.Does LangWatch have an API?
Yes, LangWatch offers an API.How do you use LangWatch?
The walkthrough on this page covers 6 steps: 1. Open the dashboard and connect a first source 2. Instrument your agent with OpenTelemetry 3. Write your first scenario 4. Add an evaluator to score the run 5. Run red teaming before users find the gaps 6. Move scenarios into CI.What platforms does LangWatch support?
LangWatch is available on Web App.What does LangWatch integrate with?
Python and TypeScript SDKs plus plain OpenTelemetry, with LangChain, LangGraph, CrewAI, Mastra, Pydantic AI, Vercel AI SDK and n8n. Coding agents Claude Code, Codex, Copilot and opencode via one command. Gateway covers OpenAI, Anthropic, Google, Bedrock, Azure, Mistral, Groq, DeepSeek and Vertex AI.What are the limitations of LangWatch?
Pricing is not published anywhere on the homepage, so budgeting requires a sales conversation. The surface area (simulations, evals, tracing, gateway, governance) is large and assumes an existing platform team to operate it. Agent frameworks are well covered, but teams on older or in-house stacks may find the integration path less smooth.
Status
StatusActive
Views0
Outbound clicks0
Added10/21/2026
Platforms
Web App
Pricing
FreemiumSubscription
