Your AI agents fail silently.
Pisama catches it and shows you the fix.
Agents return 200 and still get it wrong: loops that never break, state quietly corrupting, one bad output cascading downstream. Pisama reads every production trace, names the exact agent and step that failed, and hands you the fix. Works with the frameworks you already run.
An agent repeats the same state transitions. Every call returns 200.
A research agent calls search, gets insufficient results, rephrases, gets similar results, rephrases again. No exception, no timeout, and your dashboard stays green.
Catch it. Explain it. Fix it.
The SDK runs on your machine and finds the failure. The platform writes the patch back into your repo. Both ship in the open — see exactly which parts cost money and which don't.
It catches what you missed
Loops, hallucinated tool calls, persona drift, runaway costs, corrupted state. Detectors run on every trace, 25 externally validated at production grade. The ones tuned for structure cost nothing and never leave your machine.
It explains in plain English
Each failure comes with what broke, where it broke (the exact agent and step), and a suggested fix. No stack-trace archaeology, no eyeballing a 4k-token transcript.
It writes the fix back into your code
For LangGraph recursion limits, Pisama opens the fix as a GitHub PR you can review and merge — shipping today. Auto-fix for n8n, Dify, OpenClaw, and Anthropic Managed Agents on the roadmap. Hosted at pisama.ai.
Operations and accountability.
Pisama runs while you’re not watching. That matters to the team shipping agents and to the people who answer for them.
You can’t watch every agent run in production.
- Silent
Agent says it finished. Output is wrong. You find out from a user.
- Token burn
Loops uncontrolled while you sleep. The bill arrives later.
- Trust break
After one bad incident you start checking every turn. Automation defeated.
Your agents have no accountability layer.
- No audit
Legal asks what the agent did. The trail lives in four tools, not one.
- Scope creep
Agent acquires more agency than it was shipped with. Nobody notices.
- Cascade
One bad output poisons every downstream agent. No circuit breaker.
Detection tuned to the workflow, not just the framework.
Five narrow, domain-specific calibration packs in active R&D. Discovery stage today: no calibrated threshold, no customer traces adjudicated, published honestly as exactly that.
Accounting & Tax
Reconciliation and Posting Integrity
A bank-reconciliation workflow: human approval, authoritative ERP posting outcomes, and completion-exception state, checked against records the agent cannot edit.
Cybersecurity
Advisory Citation Integrity
Whether an agent correctly decided a published security advisory applies to a deployed asset: version-range logic, CVSS scoring, and advisory revision lineage.
Financial Risk & Compliance
SEC Filing Fact Integrity
Whether an agent's citation of a reported financial fact matches the filing itself: scale, sign, rounding, and restatement lineage against SEC EDGAR source data.
Insurance
Commercial Submission Integrity
A Commercial Auto submission workflow end to end: public source-fact lineage, broker review, and carrier transport receipts.
Legal
Research and Citation Integrity
Whether an agent's legal-research citations bind to the real source: the exact opinion, version, pinpoint, and quoted text.
Reads what you already write.
Drop-in adapters for the frameworks, runtimes, and editors you already use, plus an MCP server and generic OpenTelemetry ingestion for everything else.
Detectors for single-agent, multi-agent, and sub-agent systems. Six categories.
The named detectors, grouped by category. Each is a calibrated pattern-match against a specific failure shape, not a generic rubric. Plus framework-specific packs that know what goes wrong inside the runtimes themselves.
Planning & Decomposition
6Execution & State
7Coordination
6Verification & Quality
7Behavior & Safety
7Reasoning & Observability
5Five tiers. Heuristics first. LLMs and humans only when forced.
Fast detectors handle 90%+ of detections at zero cost. The pipeline escalates only when a tier can't conclude.
Hash
Identity matching on transition graphs. Loops, deadlocks, repetition.
Delta
Type, null, oscillation tracking. Element coverage on cross-agent payloads.
Embeddings
Behavioral embedding of outputs vs. embedding of the role.
LLM judge
Escalation tier. Invoked only when T1–T3 disagree or are ambiguous.
Human
Async review for edge cases. Optional, opt-in.
Layered by cost. The cheap checks run on every step.
Running the whole agent and reading the transcript afterward pays full price just to learn what broke. Pisama checks while the run happens, cheapest tier first, and pays for a model only where the free tiers cannot decide.
A final-output eval
Grades the finished output, pass or fail. You have already paid for the whole run, and the score cannot tell you which step broke or halt a loop while it spins.
Observability
Records every span faithfully, then decides nothing. You still have to know what to look for and rule the run good or bad yourself.
A retry
Runs it again with the same prompt and the same tools, so it meets the same failure. A retry with no diagnosis just pays twice.
The current optional metric compares aggregate process-local LLM judge spend with validation events recorded through one endpoint. Those streams are not linked by detection or cohort. It does not measure task or business outcomes, omits other review paths, and resets with the process. A durable cost-to-outcome view still requires an independent task verifier and an append-only usage ledger.
Twelve public packages. Python and TypeScript. Use what fits your stack.
The Python entry point: offline analysis, CI checks, trace scrubbing, CLI, and local MCP server.
Detection orchestrator, trace models, scoring, and registered heuristic detectors.
A standalone detector library with framework packs for LangGraph, Dify, n8n, and OpenClaw.
Zero-code auto-instrumentation for popular Python agent frameworks.
ATIF trace adapters and real-time failure hooks for agent runtimes.
Local Claude Code trace capture, token and cost tracking, exports, and opt-in sync.
Audit harnesses, datasheets, agreement metrics, and regression gates for verifiers.
The dependency-free structural detection engine behind Pisama for n8n.
TypeScript middleware, OpenTelemetry export, and PII redaction for AI SDK applications.
Composable TypeScript failure detectors used by the SDK and dashboard.
Project setup, verification, MCP, and Harbor ATIF analysis for TypeScript teams.
The n8n community node for signed execution delivery to hosted or self-hosted Pisama.
Common questions.
How is this different from observability and trace tools like Langfuse or LangSmith?
Trace stores record what your agent did and hand you the spans. You still have to know what to look for and decide something went wrong. Pisama makes that call: point it at the same traces and it names the step that failed and why, with a calibrated confidence, then gives you the fix. Tracing is the input; Pisama is the detection-and-fix layer on top. Keep your trace store and point Pisama at it.
How is this different from rubric-based LLM judges in Bedrock / Foundry / Vertex?
Those judge the artifact: was the output good? Pisama detects what happened during execution: loops, state corruption, persona drift, coordination breakdown. Different layer; complementary tools.
Is Pisama an eval tool?
No. Evals grade output quality, and they sit a layer above Pisama. Pisama detects structural process failures in running agents and reports them as binary failure modes with calibrated confidence, not generic quality scores. If you run evals, keep them; Pisama watches the layer your rubrics cannot see.
Why not just grade the final output, or retry the failing run?
Because both act too late. A final-output eval pays for the whole run first, then hands you a pass or fail with no where and no why, and it cannot stop a loop while it is still spinning. Observability records every span but decides nothing, so you are still the one reading the transcript. A retry re-runs the same prompt with the same tools and meets the same failure, paying twice for one bug. Pisama sits earlier and cheaper: the structural tiers cost $0, so it checks every step as the run happens, names the step that broke, and calls a model only in the gray zones a cheap check cannot settle. Grade the output too if you like; Pisama watches the process the grade cannot see.
Does Pisama send my traces anywhere?
The T4 LLM judge is opt-in and uses your own API key. Pisama does not proxy your model traffic, and PII redaction runs before traces are stored.
What if I'm not on a supported framework?
If your traces have transitions, shared state, and message history, the detection methods apply. We ship dedicated adapters for 12 frameworks/runtimes/editors, plus generic OpenTelemetry ingestion; anything that emits OTel (CrewAI, AutoGen, others) works out of the box.
Why heuristics over an LLM judge?
Structural process failures (loops, state corruption, coordination breakdown) have a fixed shape an LLM judge has to read and reason about every time. A heuristic detector finds the same shape in microseconds, for $0, and never truncates a long trace. Genuinely ambiguous cases still escalate to an LLM judge; heuristics handle the majority of detections first.
What does Pisama miss?
Genuinely ambiguous cases, where even careful human labellers disagree, are surfaced as advisory, not as flagged failures. We do not claim to catch everything.
Stop finding out
from your users.
Pisama catches the failures that still return 200: loops, silent corruption, scope creep, cascades. Across every framework you orchestrate.
Stop explaining to legal
what your agent did.
Signed audit trail, scope containment, regulator-grade retention. The accountability layer your customers will eventually require.