评估一个 agent 工具的成熟度,别读宣传页,读修复记录。宣传页写的是工具想成为什么;修复记录写的是它真实摔过哪些跟头、摔完之后怎么交代。这三家恰好都把失误公开着:Claude Code 记了 272 个版本的 CHANGELOG,Codex 留下写明根因的修复提交,DeepSeek Harness 干脆建了 postmortem 目录,把每起事故的根因链写成公开文档。
我把三家的失误档案各读了一遍,把能对上号的条目归了类,发现失误不是随机的:同样的几类坑,每家都踩过,只是先后不同——静默失败、错误归因、状态丢失、生命周期失控、权限绕过、上下文算错、假绿测试,七类。
这篇文章逐类拆解:每类给三个现场(版本号或提交号可查)、说清这类失误为什么在 agent 里被格外放大(提示先放在这里:因为错误信息是模型的输入,吞掉的错误会变成模型眼里的「事实」)、以及防住它的那条设计原则。文中的对照总表把七类失误的三家现场放在一张表里。
看到最后你会得到一个判断:稳定的 agent 不是设计出来的,是翻车翻出来的——准确说,是「失误被记录、被归因、变成规则和测试」这个循环跑出来的。判断一个工具值不值得托付,别问它出没出过事故,问它的事故都去哪了。
FIELD NOTES · 三份公开失误档案FIELD NOTES · THREE PUBLIC FAILURE LEDGERS
先立一个判断标准:评估一个 agent 工具的成熟度,别读它的宣传页,读它的修复记录。宣传页写的是工具想成为什么;修复记录写的是它真实摔过哪些跟头、摔完之后怎么交代。 Start with a standard for judging agent tools: skip the landing page and read the repair log. The landing page says what a tool wants to be; the repair log says what it actually broke and how it accounted for it.
Claude Code、Codex、DeepSeek Harness 这三个工具恰好都把失误公开着。Claude Code 的 CHANGELOG 记了 272 个版本,几乎每个修复都写明场景与后果;Codex 的提交史里,修复提交写明根因与约束;DeepSeek Harness 走得更远——仓库里有专门的 docs/postmortem/ 目录,四篇复盘把每起事故的时间线与根因链完整公开,外加一份开头就声明「每条规则对应一类真实翻过车或差点翻车的缺陷」的 defensive-patterns.md。这像航空业:航空安全不是靠飞机造得完美,是靠每起事故都变成全行业的习题。
Claude Code, Codex, and the DeepSeek Harness all keep their mistakes in public. Claude Code's CHANGELOG spans 272 releases, nearly every fix naming its scenario and consequence; Codex's commit history states root causes and constraints; the DeepSeek Harness goes further — a dedicated docs/postmortem/ directory publishes four incident reports with full root-cause chains, plus a defensive-patterns.md that opens by declaring every rule corresponds to a defect class that really shipped or nearly shipped here. This is how aviation works: safety didn't come from perfect airplanes; it came from every accident becoming everyone's homework.
我把三家的失误档案各读了一遍,把能对上号的条目归了类,发现失误不是随机的:同样的几类坑,每家都踩过,只是先后不同。这意味着「稳定的 agent」不是某家的独家秘方,而是一份公开的错题集——别人踩过的坑,是你免费的疫苗。 I read all three ledgers and sorted the entries that line up. The failures are not random: the same classes of pits appear in every tool, only at different times. "Stable agents" is not one team's secret recipe — it is a public workbook of mistakes, and the pits others have stepped in are your free vaccines.
失误的形状变了The shape of a failure changed
传统软件里,一个被吞掉的错误损失是局部的:这个请求失败,重试或者报错,边界清楚。agent 系统里失误会变形——因为错误信息是模型的输入。一个被静默吞掉的错误不会消失,它会变成模型眼里的「事实」:超时返回的空搜索结果,会被模型读成「代码里没有」;被静默截断的历史,会让模型以为你从没说过那句话。接下来的每一步,都建立在假事实上。 In classical software a swallowed error loses you one request: it fails, you retry or surface it, the boundary is clear. In an agent system the failure deforms — because error text is model input. A silently swallowed error doesn't disappear; it becomes a "fact" in the model's world: a timed-out search returning empty reads as "it's not in the code"; a silently truncated history convinces the model you never said that thing. Every following step stands on a false fact.
所以三家档案里最贵的失误,几乎都不是「崩了」——崩溃至少看得见。最贵的是静默地错了:系统带着错误的内部状态继续跑,直到某天有人在生产里发现,一整段工作建立在一个没人知道的坏前提上。下面七类失误,按这个标准排序展开。 So the most expensive failures in all three ledgers are almost never crashes — a crash at least shows itself. The expensive ones fail silently: the system keeps running on a wrong internal state until someone discovers, in production, that a whole stretch of work stood on a bad premise nobody knew about. The seven classes below unfold by exactly that standard.
七类失误,二十一个现场Seven classes, twenty-one scenes
01 · SILENT FAILURE静默失败:最贵的失误是没报错的错Silent failure: the priciest error is the one that never reports
- CCripgrep 搜索超时,静默返回空结果——模型看到「没有匹配」,不知道「没有」和「没搜成」是两回事。2.1.23A ripgrep search times out and silently returns empty — the model reads "no match", never learning that "none" and "didn't finish" are different things.2.1.23
- CCEdit 工具静默把弯引号替换成直引号——编辑即毁文,用户毫无察觉。2.1.47 · #26141The Edit tool silently corrupts Unicode curly quotes into straight ones — editing as vandalism, unnoticed.2.1.47 · #26141
- CXcompaction 放不下时静默丢弃父级内容,后改为 fail-closed;超大图片被跳过时,连用户附在图上的整条指令一起丢。4f2a1d8 · 09f4c45Compaction that cannot fit silently drops parent content — later made fail-closed; an oversized image skipped whole, taking its user instruction with it.4f2a1d8 · 09f4c45
- DSHbash 搜索层把下层结构化的「沙箱不可用」错误吞掉,换成通用的
SEARCH_FAILED——调用方永远查不到真因。PM-0004 第二层The bash search layer swallowed the structured sandbox-unavailable error from below and replaced it with a genericSEARCH_FAILED— the real cause unrecoverable.PM-0004, second layer
为什么在 agent 里格外贵:模型是这些错误的唯一读者,而它没有「去后台日志查一下」的能力——喂给它什么,什么就是全世界。传统系统里静默失败丢的是可用性;agent 系统里丢的是决策正确性,且损失会顺着对话复利。 Why agents amplify it: the model is the only reader of these errors, and it cannot "go check the logs" — whatever you feed it becomes the whole world. In classical systems silent failure costs availability; in agents it costs decision quality, compounding across the conversation.
防线:错误显式上抛;适配层保真传递下层结构化错误——dsh 的规则原文:an adapter must preserve structured failures owned by the seam below it。确实要吞的错误,必须写明吞了什么、为什么没人需要它(dsh 要求每个空 catch 命名它吞掉的东西)。 The defense: raise errors explicitly; adapters preserve structured failures from below — dsh's rule verbatim: an adapter must preserve structured failures owned by the seam below it. And where something must be swallowed, the code must say what it swallows and why nobody else can reach it (dsh requires every empty catch to name its catch).
02 · ATTRIBUTION错误归因:把账安在错误的头上Attribution: billing the wrong layer
- CCAPI 返回权益错误(entitlement),界面显示「Rate limit reached」——用户拿着错误的诊断去升级套餐。2.1.89An entitlement error surfaced as "Rate limit reached" — users went shopping for a bigger plan on a misdiagnosis.2.1.89
- CX服务端明确发来限流信号,客户端没把
rate_limit_exceeded分类为可重试——一次本可自愈的失败变成硬失败。e0c727d · #40931The server sent an explicit rate-limit signal; the client never classifiedrate_limit_exceededas retryable — a self-healing failure became a hard one.e0c727d · #40931 - DSH老内核上 launcher 每次执行都打一行良性提示,harness 用「前缀匹配 + 非零退出」判故障——ripgrep 正常的「没搜到,退出码 1」被误判成沙箱故障。PM-0004On older kernels the launcher prints a benign notice before every child; matching "prefix + nonzero exit" as failure reclassified ripgrep's perfectly normal no-match exit 1 as a sandbox fault.PM-0004
为什么在 agent 里格外贵:调用栈比传统软件深一层——工具在进程里、进程在沙箱里、沙箱可能在另一台机器上,每层都有输出,而输出不自带归属。归因错了,下游每一步都在错误的前提下走:模型以为沙箱坏了会绕着走该走的路,用户以为限流了会白花钱。dsh 复盘里那句结论值得贴在墙上:Process attribution requires a conjunction of independent evidence; a shared prefix is not a protocol(归因需要独立证据的合取,共享前缀不是协议)。 Why agents amplify it: the stack is one level deeper than classical software — tools inside processes, processes inside sandboxes, sandboxes maybe on another machine. Every layer emits output and the output carries no owner. Attribution wrong, everything downstream walks on a false premise: a model told the sandbox is broken detours around the right path; a user told "rate limited" pays for the wrong cure. dsh's postmortem conclusion belongs on a wall: Process attribution requires a conjunction of independent evidence; a shared prefix is not a protocol.
防线:归因要合取证据——退出码、致命行、执行记录分开采信;信息性输出与致命证据分开命名空间;分类不了的时候,宁可上报「未知原因」,也不要就近抓一个背锅的。 The defense: attribute on conjunctions of independent evidence — exit status, fatal lines, and execution records weighed separately; informational output namespaced away from fatal evidence; and when classification fails, report "cause unknown" rather than grabbing the nearest suspect.
03 · STATE & RESUME状态丢失:重启之后,还剩多少你Lost state: how much of you survives a restart
- CC
--resume静默截断近期对话(写竞态,无任何报错);异步写 transcript 失败被吞,链条一断历史蒸发;SSH 断连的退出顺序里,遥测竟排在会话数据落盘之前。2.1.77 · 2.1.91 · 2.1.51--resumesilently truncated recent history (a write race, zero warnings); async transcript writes failed silently and the chain broke; on SSH disconnect, telemetry flushed before session data.2.1.77 · 2.1.91 · 2.1.51 - CX外部连接断开时,连「这条提交到底生没生效」都不确定——修复方向是把不确定的输入隔离待人工复核,而不是假装它没发生过。a791339 · 907c34eWhen the app-server connection dropped, whether a submission even took effect was uncertain — fixed by quarantining the doubtful input for manual review, not by pretending it never happened.a791339 · 907c34e
- DSH有个简化提案建议「截断中断的轮次」,被否决——中断轮次不截断,崩溃修复会补写收尾事件,每张悬空的检查单都有回单。rejected note · repair.tsA simplification proposal to truncate interrupted turns was rejected — interrupted turns are not truncated; crash repair synthesizes closers, and every dangling order slip gets its receipt.rejected note · repair.ts
为什么在 agent 里格外贵:会话状态是 agent 的唯一记忆。丢失分两种:真丢了(数据没了)和假丢了(数据在,但重放对不上)。后者更阴险——用户看到的是「它忘了我刚说的话」,实际是账本对不上了。而长任务意味着状态积累的时间更长,丢一次的代价也更大。 Why agents amplify it: session state is the agent's only memory. Loss comes in two kinds: truly lost (data gone) and falsely lost (data present, replay won't reconcile). The second is nastier — the user sees "it forgot what I just said" when the ledger simply stopped balancing. Long tasks mean longer accumulation and a bigger price per loss.
防线:先落账再动手;写入失败必须可见,不许静默;「不知道」要显式建模——Codex 的隔离复核比假装成功诚实得多;退出顺序按价值排,遥测永远排在用户数据后面。 The defense: the ledger goes before the action; write failures must be visible, never silent; "don't know" gets modeled explicitly — Codex's quarantine-for-review is far more honest than pretending success; and shutdown orders by value, with telemetry always behind user data.
04 · LIFECYCLE生命周期失控:挂起的、僵尸的、和失灵的 Ctrl+CRunaway lifecycles: hangs, zombies, and a dead Ctrl+C
- CCMCP 调用在 SSE 断线时无限挂起——同一句话在 2.1.9、2.1.83、2.1.110 出现三次(没有看门狗);退出不杀 caffeinate,用户的 Mac 从此不睡;后台 agent 运行时 Ctrl+C 被静默忽略。2.1.9 / 2.1.83 / 2.1.110 · 2.1.83 · 2.1.49MCP calls hung indefinitely when SSE dropped — the same sentence in 2.1.9, 2.1.83, and 2.1.110 (no watchdog); quitting never killed caffeinate, so the user's Mac stopped sleeping; with background agents running, Ctrl+C was silently ignored.2.1.9 / 2.1.83 / 2.1.110 · 2.1.83 · 2.1.49
- CXWindows 上文件操作完成了,helper 进程还活着(僵尸);终态错误被立即重试、无限循环,很久之后才补上「三连败即熔断」。21c58c9 · d7510aa · 62b458cOn Windows the file operation finished while its helper process stayed alive (zombie); a terminal error restarted immediately, looping forever until a three-strikes breaker arrived much later.21c58c9 · d7510aa · 62b458c
- DSH工具全有 deadline(超时转结构化错误);子进程用独立进程组 + SIGTERM→SIGKILL 升级,宽限计时器故意不清除——主进程先退,强杀也要兑现;规则:「清理必须等到静默,不是发出请求」。timeout-policy · subprocess-local · defensive-patternsEvery tool carries a deadline (timeout becomes a structured error); subprocesses run in their own process group with SIGTERM→SIGKILL escalation and a grace timer deliberately never cleared — even if the host dies first, the kill is honored; and the rule: disposal must reach quiescence, not just request it.timeout-policy · subprocess-local · defensive-patterns
为什么在 agent 里格外贵:每个异步操作都是一笔未结算的账。挂起是「结果永远不会来」,僵尸是「活干完了但人没走」,取消失灵是「用户喊停停不下来」。共同点:全在测试覆盖最少的路径上——取消和清理是功能列表里最不显眼的行,却决定用户敢不敢把长任务交给它。 Why agents amplify it: every async operation is an unsettled account. A hang means the result is never coming; a zombie means the work finished but nobody left; a dead Ctrl+C means the user can't stop what they started. All three live on the least-tested paths — cancellation and cleanup are the least conspicuous lines in a feature list, yet they decide whether users dare hand over long tasks.
防线:谁 spawn 谁 reap;一切异步调用配 deadline + 强杀兜底;进程「退出」的判定用事实(进程树存活探测),不用希望;把「取消」当一等公民写测试。 The defense: whoever spawns, reaps; every async call carries a deadline with a forced kill behind it; "exited" is determined by evidence (process-tree probes), not hope; and cancellation gets first-class tests.
05 · PERMISSION EDGES权限边界:字符串匹配对抗图灵完备的 shellPermission edges: string matching versus a Turing-complete shell
- CC三连绕过:前缀匹配(1.0.120)、反斜杠续行(2.1.6)、被转义的 flag 被判「只读」后任意执行(2.1.98);更冷的是 2.1.78——沙箱依赖缺失时静默禁用:你以为是笼子,其实是装饰。1.0.120 · 2.1.6 · 2.1.98 · 2.1.78Three bypasses in a row: prefix matching (1.0.120), shell line continuation (2.1.6), an escaped flag judged read-only that led to arbitrary execution (2.1.98); and the coldest one, 2.1.78 — sandbox silently disabled when dependencies are missing: the cage was decoration.1.0.120 · 2.1.6 · 2.1.98 · 2.1.78
- CX提权终端的「后续输入」免审批——审批只看启动、不看续写;过期的安全评分(Guardian score)仍被用来放行动作;关掉一个工具反而意外扩大了权限。a57b398 · 035295b · b836aecFollow-up input to an escalated terminal sailed through without fresh approval — review watched the launch, not the continuation; stale Guardian scores still approved actions; disabling a tool accidentally widened access.a57b398 · 035295b · b836aec
- DSH沙箱的底线是 fail-closed:平台上没有合格隔离手段就拒绝执行,绝不返回原始命令。复盘 0002 还警告过一次险情:配置表达式未求值导致工具被静默禁用,而天真的「修复」(给它求值)反而会让默认配置意外拿到文件系统访问权。sandbox-local · PM-0002The sandbox floor is fail-closed: no qualified confinement on the platform, no execution — never the raw command. Postmortem 0002 also records a near-miss: the naive fix for an unevaluated config expression would have handed filesystem access to a default profile.sandbox-local · PM-0002
为什么在 agent 里格外贵:权限规则是字符串模式,shell 是图灵完备的语言——用模式匹配去拦一门语言,洞是补不完的。agent 又把过闸频率放大了千倍:每一次工具调用都在过安检。CC 一次静默的沙箱禁用意味着:模型带着「我受保护」的错觉在裸奔,而所有人都以为笼子是关着的。 Why agents amplify it: permission rules are string patterns; the shell is a Turing-complete language — pattern-matching against a language never runs out of holes. Agents then multiply the gate crossings a thousandfold: every tool call walks through security. One silent sandbox disable meant the model ran unconfined under the impression it was protected, while everyone believed the cage was shut.
防线:fail-closed——解析不了、依赖缺、评分过期,一律拒绝而不是默认放行;授权绑定到每次操作而不是对象(终端开着 ≠ 它后面的输入都免检);钩子和插件只能收紧、不能放宽(CC 2.1.77 的钩子降级企业 deny 规则);「配置能解析」和「配置会生效」是两个事实,分开验证。 The defense: fail-closed — unparsable, dependency-missing, score-expired: refuse rather than default-allow; bind authorization to each operation, not the object (a terminal being open does not make its future input pre-approved); hooks and plugins may only tighten, never loosen (CC 2.1.77's hooks downgrading enterprise deny rules); and "the config parses" versus "the config takes effect" are two different facts — verify both.
06 · CONTEXT ACCOUNTING上下文记账:窗口算错,模型遭殃Context accounting: a wrong window punishes the model
- CC先用到 65% 就锁死(阈值算错);后来新模型原生 1M 窗口,代码硬编码 200K——疯狂提前压缩;嵌套记忆文件被重复注入几十次;压缩后技能被对着下一条用户消息再执行一次(副作用重放)。2.1.14 · 2.1.117 · 2.1.69 · 2.1.119First, users blocked at 65% usage (threshold bug); then a native 1M-window model met a hardcoded 200K — compulsive early compaction; nested memory files re-injected dozens of times; skills re-executed against the next user message after compaction (side-effect replay).2.1.14 · 2.1.117 · 2.1.69 · 2.1.119
- CXtoken 预算按错误的模型计算;compaction 的防死循环措施写在注释里——「只要 compaction 工作良好,我们不必担心无限循环」。靠希望防守。8228e9b · codex-rs/core/src/session/turn.rs:471Token budgets computed against the wrong model; the infinite-loop guard for compaction lives in a comment — "as long as compaction works well... we shouldn't worry about being in an infinite loop". Defense by optimism.8228e9b · codex-rs/core/src/session/turn.rs:471
- DSH把压缩挂在「调档之前」的分诊关口上——省在花之前,而不是调完再精简。位置本身就是设计:等上下文进了请求再省,钱已经花出去了。compaction @ pre-stepCompaction hangs on the triage gate before history is pulled — savings happen before the spend, not after. The placement is the design: once context enters the request, the money is already gone.compaction @ pre-step
为什么在 agent 里格外贵:上下文窗口同时是三样东西——物理约束、计费单位、行为约束。算错它的后果是复合的:模型变笨(关键历史被提前压掉)、账单变高(重复注入)、任务被假性锁死(65% 就拒绝)。而每换一个模型、每加一种注入源(记忆文件、技能、工具结果),记账公式就多一个破坏点——CC 的两次事故正好是这一头一尾。 Why agents amplify it: the context window is three things at once — a physical constraint, a billing unit, and a behavioral constraint. Getting it wrong compounds: the model gets dumber (key history compacted away), the bill grows (duplicate injections), the task gets falsely locked (refusals at 65%). And every new model or new injection source (memory files, skills, tool results) adds a fresh break point to the accounting formula — CC's two accidents are exactly the two ends of that.
防线:token 记账要单一真源、按模型参数化——新模型上线必须带新的回归用例;压缩的副作用要防重放(「压缩后重执行」是一类,不是一次);注入去重要有测试锁住。 The defense: one authoritative accounting function, parameterized per model — a new model ships with new regression cases; compaction side effects need replay protection ("re-executed after compaction" is a class, not an incident); and injection dedupe gets a test to pin it.
07 · FALSE GREEN假绿测试:全绿的测试,崩真机False green: perfect tests, broken product
- DSH最典型的单例:178 个单测全绿、100% 行覆盖率,真实编辑器接上第一个请求就崩。根因有三层——多导出让加载器丢掉关键元数据;测试全部绕过真实装载路径;唯一的端到端测试被 API key 门禁挡在 CI 外。复盘原句:Coverage proves lines ran; it says nothing about whether the feature works the way it ships.PM-0001The canonical single case: 178 unit tests green, 100% line coverage, and the first request from a real editor crashed. Three layers deep — a multi-export made the loader drop key metadata; every test bypassed the real loading path; the only e2e was key-gated out of CI. Verbatim: Coverage proves lines ran; it says nothing about whether the feature works the way it ships.PM-0001
- CC
--resume的缓存回归从 2.1.69 存活到 2.1.90——21 个版本没人发现,因为没人测「重启后打开旧会话」这条真实路径;AVX 崩溃在 2.1.17 和 2.1.19 被原样修了两遍。2.1.69 → 2.1.90 · 2.1.17 / 2.1.19A--resumecache regression lived from 2.1.69 to 2.1.90 — 21 releases, because nobody tested "reopen an old session after restart"; an AVX crash was fixed identically twice, in 2.1.17 and 2.1.19.2.1.69 → 2.1.90 · 2.1.17 / 2.1.19 - CX一个「防丢失」修复引入前台死锁,代码注释明说要「照抄 remote-client 那边的补丁」——补丁生补丁;Guardian 历史保护一周修三次,
update_plan的默认开关先开后关。app-server-client/src/lib.rs:334 · #41846/#41857/#41858A data-loss fix introduced a foreground deadlock, with the comment instructing to "match the remote-client fix" — a patch patching a patch; Guardian history protection fixed three times in a week; update_plan's default flipped on then off.app-server-client/src/lib.rs:334 · #41846/#41857/#41858
为什么在 agent 里格外贵:agent 工具的「真机」特别难模拟:真实模型、真实终端、真实文件系统、真实用户的随时打断。测试替身模拟得越像,盲区越一致——所有替身都没模拟的那个细节,就是全绿测试集体漏掉的细节。而 agent 的失败模式又偏爱这些路径(取消、重启、断连),于是「测试全绿」和「能上生产」之间的缝,比传统软件宽得多。 Why agents amplify it: an agent tool's "real machine" is brutal to simulate: real models, real terminals, real filesystems, real users interrupting at will. The better the test double mimics reality, the more aligned the blind spots — the one detail no double simulates is the detail every green test misses together. And agent failures favor exactly those paths (cancel, restart, disconnect), so the gap between "tests green" and "production ready" is wider than in classical software.
防线:至少留一条无门禁、走真实装载路径的冒烟链路(dsh 在 PM-0001 之后新增的正是这个);覆盖率别当正确性用;每个修复带一个能抓住它的回归测试——否则你不是在修 bug,是在给 bug 排队。 The defense: keep at least one ungated smoke path through the real loading route (exactly what dsh added after PM-0001); never mistake coverage for correctness; and every fix carries a regression test that would have caught it — otherwise you are not fixing bugs, you are scheduling them.
补丁生补丁:失误的第二定律Patches patching patches: the second law
三家档案里还有一条暗线:修复本身也会成为失误源。CC 2.1.110 给限流重试加了上限,2.1.111 整体 revert,理由很诚实——「它用更长的等待换了更多彻底的失败」;「Always Allow」生成的坏权限规则连修三次(2.1.47 → 2.1.72 → 2.1.77);Codex 那个防丢失变死锁的修复是同类;dsh 的 PM-0001 里,修掉第一个 bug 后才暴露第二个——「优雅的影子解释是真实存在的,但它是第二个 bug」。 A second thread runs through all three ledgers: the fixes themselves become failure sources. CC 2.1.110 capped rate-limit retries and 2.1.111 reverted the whole thing, with an honest reason — "it traded long waits for more outright failures"; the bad rules generated by "Always Allow" took three fixes (2.1.47 → 2.1.72 → 2.1.77); Codex's data-loss fix becoming a deadlock is the same species; and in dsh's PM-0001, fixing the first bug revealed the second — "the elegant shadow explanation was real but was the second bug."
这不是哪家工程水平的问题,是复杂系统的固有摩擦:每次修复都在改变系统的行为假设,而系统的其他部分建立在旧假设上。所以判断成熟度还有一条隐藏指标:看 revert 和「修复的修复」的密度。没有 revert 记录的工具不是没犯错,是没诚实记录。 This is not a competence problem; it is the intrinsic friction of complex systems: every fix changes the system's behavioral assumptions while the rest of the system still stands on the old ones. Which yields a hidden maturity metric: the density of reverts and fixes-of-fixes. A tool with no revert history isn't one that never errs — it's one that doesn't record honestly.
FIG · 对照总表FIG · THE LEDGER, SIDE BY SIDE
七类失误 × 三个工具:公开现场总表Seven failure classes × three tools: the public scenes
同一类坑,三家各踩一遍。版本号与提交号均可复核。The same pit, stepped in by all three. Versions and commits are checkable.
| 失误类别Class | Claude Code | Codex CLI | DeepSeek Harness |
|---|---|---|---|
| 静默失败Silent failure没报错的错最贵the unreported error costs most | 搜索超时返回空结果;弯引号被静默替换timed-out search returns empty; curly quotes silently replaced2.1.23 · 2.1.47 | compaction 静默丢弃;大图连累整条指令compaction silently drops; oversized image eats its instruction4f2a1d8 · 09f4c45 | 结构化沙箱错误被吞成通用失败structured sandbox error swallowed into a generic failurePM-0004 |
| 错误归因Attribution账安在错误的头上billing the wrong layer | 权益错误显示成限流,用户误升级套餐entitlement error shown as rate limit; users bought the wrong cure2.1.89 | 限流信号未被分类为可重试rate-limit signal never classified as retryablee0c727d | 子进程正常退出被判成沙箱故障a child's normal exit blamed on the sandboxPM-0004 |
| 状态丢失State & resume重启之后还剩多少你how much of you survives a restart | --resume 静默截断;遥测排在数据落盘之前--resume silently truncates; telemetry flushed before session data2.1.77 · 2.1.51 | 断连后「是否已提交」不确定,草稿隔离待复核after disconnect, submission state unknown; drafts quarantined for reviewa791339 · 907c34e | 拒绝「截断中断轮次」提案,改为补写收尾truncating interrupted turns rejected; closers synthesized insteadrejected note · repair.ts |
| 生命周期Lifecycle挂起、僵尸、失灵的取消hangs, zombies, dead cancels | MCP 挂死三修;caffeinate 不死;Ctrl+C 失灵MCP hangs fixed thrice; caffeinate outlives quit; Ctrl+C ignored2.1.83 · 2.1.49 | Windows helper 僵尸;终态错误无限重试Windows helper zombie; terminal error retried forever21c58c9 · d7510aa | deadline + SIGKILL 兜底;dispose 等到静默deadline + SIGKILL backstop; disposal reaches quiescencetimeout-policy · subprocess-local |
| 权限边界Permission edges字符串匹配 vs 图灵完备string patterns vs Turing-complete | 前缀/续行/转义三连绕过;沙箱静默禁用prefix/continuation/escape bypasses; sandbox silently disabled1.0.120 · 2.1.98 · 2.1.78 | 提权终端续写免审;过期分数放行动作escalated terminal's follow-ups unreviewed; stale scores approvea57b398 · 035295b | 无隔离即拒绝(fail-closed);天真修复险扩权no confinement, no execution (fail-closed); naive fix nearly widened accesssandbox-local · PM-0002 |
| 上下文记账Context accounting窗口算错,模型遭殃wrong window, punished model | 65% 锁死;200K 硬编码遇 1M 模型;重复注入blocked at 65%; 200K hardcoded for a 1M model; duplicate injections2.1.14 · 2.1.117 · 2.1.69 | 预算按错误模型算;死循环防线写在注释里budget on the wrong model; loop guard lives in a comment8228e9b · turn.rs:471 | 压缩挂在调档之前——省在花之前compaction before the pull — savings before the spendcompaction @ pre-step |
| 假绿测试False green全绿的测试,崩真机green tests, broken product | --resume 回归存活 21 个版本;AVX 崩溃修两遍--resume regression lived 21 releases; AVX crash fixed twice2.1.69 → 2.1.90 | 防丢失修复引入死锁,抄隔壁补丁再补data-loss fix deadlock; patched again from the neighbor's fixlib.rs:334 | 178 全绿 + 100% 覆盖率,真实连接即崩178 green tests + 100% coverage; first real connection crashedPM-0001 |
失误制度:三家真正共享的东西The mistake institution: what the three actually share
把档案摆在一起看,比任何单个失误更重要的是三家的「失误制度」——失误被怎样对待。dsh 是制度化最完整的:事故进 docs/postmortem/(时间线、根因链、影响全公开),根因凝结成 defensive-patterns.md 的七条规则,规则被列为写代码前的强制阅读,修复配套无密钥的回放场景锁死。失误在它那里是一条流水线:事故 → 复盘 → 规则 → 测试 → 下一次事故更难发生。
Read side by side, what matters more than any single failure is how each institution treats failure. dsh is the most complete pipeline: accidents enter docs/postmortem/ (timeline, root-cause chain, impact — all public), root causes condense into the seven rules of defensive-patterns.md, the rules are required reading before lifecycle code, and fixes land with keyless replay scenarios pinning them. Failure runs an assembly line there: incident → postmortem → rule → test → a harder next accident.
Claude Code 的制度是 changelog 本身:272 个版本,每个修复都交代场景与后果——更快的循环,更轻的仪式。Codex 的制度藏在提交纪律里:修复提交写明根因与约束(「过期分数不能放行」「超时必须 fail closed」),关键防线直接写成源码注释里的设计守则。形态不同,结构相同:失误 → 档案 → 规则或测试 → 更难的下次失误。 Claude Code's institution is the changelog itself: 272 releases, every fix stating its scenario and consequence — a faster loop with lighter ceremony. Codex's institution hides in commit discipline: fixes state root cause and constraint ("stale scores cannot approve actions", "fail closed on timeout"), and key defenses live as design rules in source comments. Different forms, identical structure: failure → record → rule or test → a harder next failure.
给自己建 agent 的人:四步失误制度Building your own: a four-step mistake institution
如果你正在搭自己的 agent 系统,最值得从三家抄的不是任何机制,而是这个循环本身。落到四步: If you are building your own agent system, the thing worth copying from these three is not any single mechanism — it is the loop itself. Four steps:
- 01记:每次翻车留下记录,一行也行,但要持久、可搜索。别用「下次注意」当记忆——那是三家档案证明最先腐烂的东西。Record: every incident leaves a trail — one line is fine, but durable and searchable. "We'll be careful next time" is the first thing these ledgers prove rots.
- 02归:归因到「哪个设计假设错了」,不是「谁手滑」。dsh 的 0004 号复盘整个就是在修「归因」——错误归因本身就是事故。Attribute: trace to which design assumption was wrong, not who slipped. dsh's postmortem 0004 is entirely about repairing attribution — misattribution is itself the accident.
- 03变:每条失误变成一条规则或一个测试。只写不测的规则会腐烂,只测不写的测试没人知道在守什么。修复必须带一个能抓住它的回归测试。Compile: every failure becomes a rule or a test. Rules without tests rot; tests without rules guard something nobody can name. And every fix carries the regression test that would have caught it.
- 04读:定期读别人家的失误档案。Claude Code 的 CHANGELOG、Codex 的提交史、dsh 的 postmortem 目录都是公开的——同一个坑三家各踩一遍的样子,是最好的教材。Read: other people's failure ledgers are public — Claude Code's CHANGELOG, Codex's commits, dsh's postmortems. The same pit stepped in three different ways is the best textbook there is.
docs/postmortem/ 目录。
Boundary: the three ledgers have different granularity — changelog, commits, and postmortems are different record forms. This piece only takes the classes that line up across tools and is not a ranking of engineering quality. Versions, commit hashes, and postmortem numbers cited are all public; dsh's four reports live in its repo under docs/postmortem/.
YOLOLAB · FIELD NOTES ON PUBLIC FAILURE LEDGERSYOLOLAB · FIELD NOTES ON PUBLIC FAILURE LEDGERS