Agent failure modes, defined

Task specification, decomposition, resource allocation, workflow design.

• Specification Mismatch

  F1 0.70
  Detects when task output doesn't match the user's original specification. Catches scope drift, missing requirements, language mismatches, and conflicting specifications.
• Poor Task Decomposition

  F1 0.73
  Detects when task breakdown creates subtasks that are impossible, circular, vague, too granular, or too broad. Critical for complex multi-step agent workflows.
• Resource Misallocation
  Detects when multiple agents compete for shared resources, leading to contention, starvation, or deadlock. Common in parallel multi-agent architectures.
• Inadequate Tool Provision
  Detects when agents lack the tools needed to complete assigned tasks. Catches hallucinated tool names, missing capabilities, and suboptimal workarounds.
• Flawed Workflow Design

  F1 0.80
  Detects structural problems in agent workflow graphs including unreachable nodes, dead ends, missing error handling, bottlenecks, and missing termination conditions.

Derailment, withholding, coordination, and communication breakdown.

• Task Derailment

  F1 0.82
  Detects when an agent goes off-topic or deviates from its assigned task. One of the most common failure modes (20% prevalence in MAST-Data).
• Context Neglect

  F1 0.87
  Detects when an agent ignores or fails to use upstream context provided by previous agents or workflow steps. Critical in multi-agent handoffs.
• Information Withholding

  F1 0.87
  Detects when an agent doesn't share critical information with peers, including omitting negative findings, over-summarizing, or selectively reporting.
• Role Usurpation
  Detects when an agent exceeds its designated role boundaries, taking actions or making decisions reserved for other roles.
• Communication Breakdown

  F1 0.82
  Detects when messages between agents are misunderstood or misinterpreted, causing incorrect downstream behavior.
• Coordination Failure

  F1 0.80
  Detects handoff failures, circular delegation, excessive back-and-forth, and ignored messages between coordinating agents.

Output validation, quality gates, completion misjudgment, retrieval quality.

• Output Validation Failure
  Detects when validation steps are skipped or bypassed, or when approval is given despite failed checks.
• Quality Gate Bypass
  Detects when agents skip mandatory quality checks, ignore quality thresholds, or proceed despite failing checks.
• Completion Misjudgment

  F1 0.74
  Detects when an agent incorrectly determines task completion, including premature claims, partial delivery, and ignored subtasks. Most prevalent failure mode (40% in MAST-Data).

Behavioral patterns that span planning, execution, and verification: loops, persona drift, hallucination, injection, state corruption.

• Loop Detection

  F1 0.85
  Detects when agents get stuck repeating the same sequence of actions. Uses multiple detection methods from hash-based to semantic clustering.
• Context Overflow

  F1 0.82
  Detects when agent context windows are approaching or exceeding capacity, causing information loss and degraded performance.
• Prompt Injection

  F1 0.94
  Detects prompt injection attacks and jailbreak attempts targeting LLM agents. The highest-accuracy detector in the system.
• Hallucination

  F1 0.77
  Detects factual inaccuracies, fabricated information, and unsupported claims in agent outputs.
• Grounding Failure

  F1 0.67
  Detects when output contains claims not supported by source documents. Agents achieve less than 45% accuracy on document-grounded tasks (OfficeQA benchmark).
• Retrieval Quality

  F1 0.82
  Detects when agents retrieve wrong, irrelevant, or insufficient documents for a task. Retrieval is the primary bottleneck in RAG systems.
• Persona Drift

  F1 0.93
  Monitors when agents deviate from their intended role, personality, or behavioral constraints over time. Uses role-aware thresholds for different agent types.
• State Corruption

  F1 0.91
  Detects when agent memory or state becomes corrupted, including type drift, schema violations, nullification, and velocity anomalies. Second-highest accuracy detector.
• Cost Tracking
  Tracks token usage and estimated costs across 30+ LLM models. Alerts when costs exceed budgets or usage patterns suggest inefficiency.