Completion Misjudgment

Detects when an agent incorrectly determines task completion, including premature claims, partial delivery, and ignored subtasks. Most prevalent failure mode (40% in MAST-Data).

Examples

• Agent claims "all 10 endpoints documented" but only 8 are present
• Task marked complete with "planned for future work" items still pending
• JSON output has status: "complete" but documented: false for key items
• Agent delivers 80% of requirements and declares the task done

Detection methods

Completion Markers

Identifies explicit and implicit completion claims

Quantitative Check

Verifies numerical completeness ("all", "every", N items)

Hedging Detection

Flags qualifiers like "appears complete" or "seems done"

JSON Indicators

Checks structured output for incomplete flags

Calibration accuracy

From the Pisama calibration set. See detector scoreboard for the full table.

Subtypes

• premature completion
• partial delivery
• ignored subtasks
• missed criteria