DeepSeek published a design essay and a companion research prototype; both sit on top of the already-open DeepSeek Harness. Most summaries merge them into one product. The boundary between them is the most useful part.
The prototype is a narrow, MIT-licensed research implementation for one GitHub repository. Its README says it is not an official DeepSeek product and that some designs in the essay are future directions, not shipped code.
The reusable idea is not unlimited autonomy. It is a control plane that treats human attention and authorization as scarce: deterministic gate, model assessment, immutable mandate, ticket, assignment, durable outbox, and recovery before retry.
This article maps that control plane onto an AI image/video product, then packages the result as a media-generation harness, a SKILL.md for coding agents, and a YAML contract for review.
Research boundary · 2026-11-02
Three things were released, not one
On 31 October 2026, the DeepSeek Harness team published a design essay titled Exploring Super-agent Design: Towards a More Autonomous Agent System and a companion research prototype, dsh-superagent-prototype. Both sit on top of the already-open DeepSeek Harness. Most coverage collapses them into “DeepSeek launched a super agent”. The boundary between them is the most useful part.
Here is the separation. DeepSeek Harness is the substrate: a Cordis-based “everything is a plugin” harness that runs as a desktop app or as a web app started from code. It provides sessions, tools, skills, automation and workspaces. Super-agent is the system shape proposed by the essay: continuous across sessions, concurrent across tasks, proactive about events, and durable over time. The prototype is a deliberately narrow reference implementation of that shape: one GitHub repository, optional Feishu, MIT licensed.
| Artifact | What it is | What it is not |
|---|---|---|
| DeepSeek Harness | Open-source agent harness with a plugin architecture; runs on desktop and web; provides Session, tools, skills, automation and workspace. | Not Super-agent itself, and not a generic “multi-agent framework”. |
dsh-superagent-prototype | A proactive coding agent for one GitHub repository, packaged as a DSH bundle. Each work item gets its own Session and directory; results return to the original conversation, for example as a pull request. | Not an official product. The README calls it a research prototype, adapted for publication and not identical to the internal version. |
| Design essay | An architecture essay: Router/Worker, event-driven operation, asynchronous interaction, compute/state/environment separation, Watchdog. | Not product documentation or a complete prototype spec; some designs are future directions. |
If you are new to the harness itself, two earlier teardowns cover the life of a request and eight lines of defense; this piece does not repeat them. It focuses on the Super-agent design and the new prototype.
Evidence note: the essay was published on the DeepSeek Harness team’s WeChat account, and the original link is not stably fetchable. I read Machine Heart’s full Chinese translation and cross-checked every mechanism against the prototype’s docs/ and src/. The prototype was inspected at commit 6337fc8 (2026-10-31): roughly 92 .ts and 18 .tsx files under src/, 65 files under tests/, 9 design documents and 15 prompt files. Where the essay and the code diverge, I say so below.
The bottleneck is not capability. It is attention.
DeepSeek’s observation is concrete: a person can usually keep only three to five active sessions in their head. As models get better, the constraint moves from “can the model finish the task?” to “can the human remember, switch between, and review the tasks?” The essay names four properties of the proposed shape:
Continuous
The user works in familiar surfaces such as IM and GitHub comments and feels one agent. A plan agreed in chat and then changed in a PR review is still the same piece of work.
Efficient
Interaction is fast, several complex tasks run in parallel, and a long task does not block the next instruction.
Proactive
It watches environment events instead of waiting for a command. It acts when action is warranted and stays quiet otherwise.
Durable
It stays online, connects to the real work environment, and recovers from failures instead of disappearing when a session ends.
One sentence in the essay deserves to be copied on its own: a session should be an implementation detail, not a user’s cognitive burden. That is why the two-tier design exists. It is not meant to make the agent look busy; it moves context maintenance and concurrency out of the human’s working memory.
The first principle follows from that: take as little human attention as possible, and give as much value as possible. Every gate, mandate, ticket and panel in the prototype is an engineering expression of that principle.
Five modules: how the system actually runs
1. Router / Worker: continuity on one side, throughput on the other
The Router is a resident global layer. It receives human interaction, maintains global context and dispatches work. It does not execute tasks itself and may not even have a Bash tool. Workers are context-isolated sessions with the full execution toolset; each focuses on one task. Workers do not talk to the human directly. They report state and results to the Router, which owns the external reply.
Why is a single Router session not automatically a throughput bottleneck? Triggers that arrive during one inference turn queue up and are merged into the next turn. Router steps are usually long-input, short-output; merging several Triggers into one call lets prefill compute them in parallel, so request latency grows much more slowly than batch size. The essay’s stress test at roughly 1 to 40 Triggers per minute supports this: batching lowers the inference time allocated per decision.
2. Event-driven: a fast path filters, a slow path judges
Environment events are high-noise and low-trigger. Calling a model for every message is too slow and too expensive. The first layer is deterministic code: source, group, user and bot allowlists, whether the agent was addressed. The second layer is the model: is this message actually a request to implement something? That is the fast-path / slow-path split.
After filtering comes routing. Once a Thread is owned, later events go directly to the Worker subscribed to that Thread; they do not pass through the Router again. Only unowned events return to the Router. In the essay’s experiment, removing this direct path increased Router decisions by about 30% and cached read tokens from 23.23 million to 31.77 million, a 37% increase.
Context is managed with two operations: Inline puts recently useful information directly into the window; Spill keeps low-frequency or over-budget content outside the window and leaves a reference or summary. Objects carry IDs, and each injected object inlines the IDs of the next related objects. The model can then walk the reference graph on demand. The essay calls this chain of recall.
One step further is speculative execution: when the agent suspects something may need work, it tries in the background. If the result is useful, it delivers; if not, it stays silent. The essay’s example is a bug report in a group chat: the agent reproduces it and prepares a fix in the background, then opens a PR only after confirming the problem and finding a simple fix.
3. Human interaction: full duplex and channel-specific output
The classic ReAct loop is half-duplex: a synchronous tool call blocks the loop until it returns, and new input waits. Super-agent changes long-running commands to asynchronous start() / stop(). start() immediately returns “Started”; when the background work finishes, its result arrives as a new event in the session, while the foreground keeps answering the human. The essay says this is already implemented in DeepSeek Harness but not enabled by default.
Output must match the channel. IM is low-distraction and lightweight: notify only when a decision or review is needed, and keep a single reply under 100 words. GitHub, email and documents are asynchronous and high-density: write the background, the change and the verification in the project’s format. Worker reasoning and tool calls never stream into external channels, which prevents information overload in team settings.
4. Compute, state and environment separation
A local agent usually binds three things to one machine: the compute that advances the agent loop, the session state that must persist, and the repository and toolchain that form the execution environment. When the device sleeps or goes offline, the agent stops. Super-agent separates them in two steps. First, make the loop stateless and put session state in a database, so the agent is no longer tied to one machine’s lifetime. Second, let the cloud loop reach the execution environment through a lightweight daemon on the edge that stores no session state. This is the BYOM, or Bring Your Own Machine, idea.
It addresses a real problem: environments are hard to copy. Uncommitted local changes, private credentials, intranet services, specific hardware and toolchains are expensive to reproduce in a cloud sandbox and easy to get wrong. Keeping the environment where it is, and passing only commands and results across the boundary, is more realistic than pretending the environment can be cloned.
5. Watchdog: an external control loop
A long-running system cannot rely only on its own internal logic for reliability. The essay puts an independent Watchdog session outside the main system, driven by scheduled messages. It checks infrastructure failures and state-recovery failures, and uses a model to judge whether each Worker is making real progress or drifting from the goal. It intervenes when it finds a problem and keeps checking transient external failures.
There is an easy-to-miss side effect: the Watchdog is itself a full agent session with tools. Maintainers can ask it in natural language why something failed, and can let it change prompts or trigger rules online. That is powerful, and it means the Watchdog must be designed as a high-privilege component, not as a read-only monitoring script.
What the prototype actually implements: an authorization operating system
The essay describes the full shape; the prototype implements a narrow slice. That slice happens to be the reusable part: it turns “this work may be done” into a data structure, then hangs execution, retry, recovery and notification underneath it.
The prototype’s main table has eleven concepts: Source, Trigger, Thread, Correlation, Channel, Policy, Eval, Mandate, Ticket, Assignment and Subscription. Below are the ones that matter most for a product; Decision is Eval’s output and gets its own row.
| Concept | Definition in the prototype | Why it matters |
|---|---|---|
| Source | A stream of events. It holds its read position and bot identity, and drops its own messages. | A restart must not replay events. The source also defines who may originate work there. |
| Trigger | One immutable event with source metadata such as the actor. | The unit of audit and idempotence. Accepting the same Trigger twice creates one Ticket. |
| Thread | Triggers in time order; the source decides which Triggers belong together. | Gives Eval a consistent view. At most one Eval runs on a Thread at a time. |
| Eval | A judgement on a batch of Triggers, in two layers: gate and assessment. | Decides whether new work starts, continues or stops. It holds no state. |
| Decision | The output of Eval: accept, stop, ignore or unavailable. | Every decision is replayable, including failure. |
| Mandate | What an accept grants: Policy version, scope, strength, matched rules, sender identity and how the source received the Trigger. | Immutable. Wider scope means a new Ticket. It answers why the work may be done, how far it may go, who approved it and under which rules. |
| Ticket | A unit of authorized work with its own lifecycle. | Only Eval creates one. A human follow-up can enter an existing Ticket without a new Eval. |
| Assignment | The binding of a Ticket to one Executor instance. | A Ticket can have many Assignments over time. A retry does not fork the ticket or the bill. |
| Subscription | A many-to-many relation between Tickets and Threads. | It provides context only and grants no new authority. This is the defense against context contaminating authorization. |
| Channel | The write side of a Thread: replies, PRs, reviews. | Results return to where the request happened. Messages stay ordered per Ticket and conversation. |
| Policy | Versioned assessment rules, accept mode, and each Source’s allowlist and admission table. | Every Mandate records the version it relied on. A policy change does not immediately revoke in-flight work. |
Gate and assessment: the default answer is “no”
The gate is code: does this person have the authority to ask the agent for work? The assessment is a model: is this message actually a request to implement something? Each Source has an admission table, and its defaults are revealing:
| Situation | Default | Meaning |
|---|---|---|
| On the allowlist, but does not address the agent | inferred | A candidate only; it does not start work automatically. |
| Not on the allowlist, but mentions the agent | ignore | A stranger mentioning the agent is ignored by default. |
| Not on the allowlist and does not mention the agent | ignore | Ignored. |
The accept mode matters even more:
| Mode | Effect | When to use it |
|---|---|---|
off | No inferred candidate counts. | Proactive work fully disabled. |
observe (default) | Candidates are assessed; a positive assessment is recorded as “would accept” but nothing is accepted. | See what it would do before granting authority. |
auto | A positive assessment accepts the work. | After reviewing the observe log and confirming the behavior. |
That order is the reverse of how most teams deploy automation: not “turn it on and switch it off after an incident”, but “let it write a diary first, read the diary, then grant authority”. Under the default configuration, the agent does no proactive work at all.
Mandate vs Subscription: authority and context are separate
This is the single most valuable idea to copy. A Ticket can subscribe to any number of Threads for context. Messages from a subscribed Thread go straight into the execution session, bypassing the gate but granting no new authority. The Ticket remains limited by its Mandate.
That solves a practical problem. The agent opens a PR; people discuss it, CI fails, a reviewer leaves comments. Those messages should reach the agent, but they should not be treated as new work requests that go through admission again. Conversely, if a message is subscribed by several Tickets, it fans out to all live Tickets; but “who may ask the agent to work” is always decided by the Mandate, never by the subscription.
Only three operations change a Mandate: accept creates it, a person’s explicit stop withdraws it, and a failed revalidation against the current Policy revokes it. A policy change does not immediately cancel running Tickets; it delivers the change as an Input, so work is not left half-finished. That is a product decision: tolerate an in-flight task finishing under the old rules rather than create an interruption.
There is also a deliberately restrained choice: a person is not a domain concept. A Trigger carries the actor identifier the Source gives it. The Feishu allowlist belongs to the Feishu Source; the GitHub allowlist belongs to GitHub. The fact that one person has different accounts on different platforms lives in team prompt text, which the agent reads when needed. The cost is that identity consistency is left to natural language; the benefit is that the system does not maintain a global identity table that will always drift.
Execution and recovery: failure is a handoff protocol
Each Ticket gets one execution Session named worker-<ticket id> and one directory. A prepare command, typically git worktree, creates the checkout. Foreground shell commands move to the background after about five seconds so the worker can receive new messages at its next step. Background work has at most about 30 minutes per turn; at the limit, shell jobs are killed but subagents may continue. A handoff to a person frees the worker slot immediately. Beyond the configured concurrency limit, one extra slot is reserved for a person continuing an existing Ticket, so new work cannot starve a human follow-up.
Ticket stages are not a coarse “running / done”. They are defined by who the Ticket is waiting for: person / delivery (input delivery failed), person / answer (waiting for an answer), person / review (waiting for review), person / blocked, failed, working / queued, idle (waiting for a Trigger), and working. Every person/* stage means the Ticket cannot continue without human input. The panel’s first question is therefore: is anything waiting for me?
Recovery is equally concrete. After a restart, a running Assignment ends with “host restarted”; the new Assignment starts with a recovery note telling the Executor to check its directory, git history and GitHub before repeating any external action. Every external effect is stored in an outbox first and performed after commit. Failed deliveries back off from 2 seconds, doubling to 1 minute. Channel messages stay ordered per Ticket and conversation. None of this is glamorous, but it is what prevents a crash from becoming a duplicate charge or a duplicate submission.
The evidence: 12.4 minutes is a scheduling result
The essay reports an initial experiment: about 300 messages, of which about 20 are work requests; the rest are chatter, follow-ups, stop messages and CI events. Coding work is replaced by a tool that only consumes time. Each configuration runs five times. The experiment measures scheduling and collaboration, not code-generation quality.
| Configuration | p50 completion | p95 completion | How to read it |
|---|---|---|---|
| Super-agent | 12.4 min | 15.7 min | Two tiers, direct subscription and batching. |
| Single session + subagent (neutral prompt) | — | 29.5 min | In all five runs the model never called the subagent; work stayed in the main session and was interrupted by new messages. |
| Single session + subagent (delegate-first prompt) | — | Still slower than Super-agent | Four of five runs delegated, but one judged the work “small” and did it inline, producing a long tail. |
The second result is the value of the direct path. Removing subscription-based direct delivery leaves completion time roughly unchanged, but Router decisions increase by about 30% and total cached read tokens rise from 23.23 million to 31.77 million, a 37% increase. Direct delivery does not improve quality or speed in this workload; it saves the Router a substantial amount of tokens and inference.
Do not extrapolate this number. It is a synthetic workload, a simplified environment, one team and five repetitions; the coding tool only consumes time and produces no code. It supports “the scheduling layer deserves separate design”. It does not prove “Super-agent is always twice as fast”, and it says nothing about your providers, failure rates or cost structure. You have to run that experiment yourself.
Essay vs prototype: what you can copy today
This is the most important table in the article. Many summaries present the essay’s future designs as shipped features. For someone building a product, that creates exactly the wrong expectations.
| Design | Essay | Prototype | Recommendation |
|---|---|---|---|
| Router / Worker | Yes | Yes: main Session plus one Executor Session per Ticket | Copy the responsibility split. |
| Two-level filter (gate + assessment) | Yes | Yes: code gate plus model assessment | Copy it, especially the observe default. |
| Direct subscription | Yes | Yes: Subscription fan-out bypasses the gate | Copy it, but also copy “grants no authority”. |
| Mandate / Policy version | Yes: authorization and rule principles | Yes: immutable Mandate records the Policy version | Copy it. This is the most valuable part. |
| Durable state / recovery | Yes: compute-state separation, long-running operation | Yes: effects stored first, restart with a recovery note | Copy it. Media sites need it even more. |
| Exception / human-attention panel | Yes: low distraction, channel-specific output | Yes: person/* stages come first | Copy it. |
| Speculative execution | Yes | No complete implementation found | Do not copy yet; full video renders are too expensive. |
| Async ReAct loop | Yes; implemented in DSH but not default | Depends on DSH; the prototype only moves foreground shell to background after 5s | Check the DSH version; do not assume it is on. |
| Inline & Spill / chain of recall | Yes | Partial: truncation plus read pointers | Borrow the idea, but do not claim the prototype fully implements it. |
| Compute / state / environment separation | Yes | No complete cloud/edge split; execution stays in the DSH environment | A media site can implement it with a cloud queue, object storage and stateless workers. |
| Watchdog | Yes: independent Session | No independent Watchdog found | Start with deterministic checks, then consider a model. |
Nine lessons for an AI video/image site
These are my translations into media generation. They are not DeepSeek’s text; DeepSeek is describing a general agent system, not a video product.
- Model a generation request as a ticket, not a chat message. A request needs an immutable job id, an authorization scope and a billing subject. Chat is context, not the state machine.
- Money is a deterministic gate, not a model judgement. Balance, quota, model/resolution/duration allowlists, per-request cost ceiling and pre-moderation are code. The model may answer “is this prompt clear or risky?” It may not set price or permission.
- Separate authority from context. “Change the background to a beach” in the same project is context and can enter the existing job. “Go from 720p to 4K and from 5 to 10 seconds” expands scope and needs a new quote and a new mandate. Chat cannot raise the budget.
- Separate Ticket from Attempt. One job can have many provider attempts. A retry does not fork the order, the bill or the user-visible job id. Each attempt records provider, provider job id, request hash and raw response.
- Record before the side effect; reconcile before retry. Reserve credits, write the outbox row, then submit. If provider state is unknown, query it; do not resubmit. After a restart, reconcile before deciding to settle or release.
- Give every job its own workspace and asset prefix. Temporary files, references, outputs and thumbnails are isolated by job id. A provider URL is not storage; mirror it to your own R2/S3 and verify checksum, dimensions, duration and moderation.
- Expose four states; interrupt only on exceptions. Working, Needs you, Ready, Failed. Do not stream provider polling, retries or chain-of-thought into the main UI; put them in a developer trace.
- Downgrade speculation. DeepSeek’s speculative execution assumes background compute is relatively cheap. Full video rendering is not. Speculate on prompt cleanup, safety pre-check, cost estimate, thumbnails and 480p drafts; do not speculatively render full 1080p/4K by default, and never create a user-visible charge for it.
- Observe a new provider before auto-routing to it. Shadow mode: quote and record “would route”, but do not submit. Compare cost, latency, failure rate and human review quality, then roll out by percentage. That is the prototype’s accept mode applied to provider routing.
A mapping table: from agent concepts to media generation
| DeepSeek concept | Media-site equivalent | Rule to keep |
|---|---|---|
| Source | Web upload, public API, Stripe, moderation callback | Dedupe by source event id; store before processing. |
| Trigger | One generation request / payment callback / moderation result | Immutable; a client retry must hit the same job. |
| Thread | Project, conversation, asset lineage | Provides context; never grants budget. |
| Gate | Auth, credits, catalog, resolution/duration, cost ceiling, pre-moderation | All deterministic; return one actionable error. |
| Assessment | Prompt clarity, reference suitability, policy risk | Advice only; cannot change price, credits or permission. |
| Decision | accept / clarify / reject / defer | clarify asks one concrete question. |
| Mandate | Model, resolution, duration, max cost, fallback list, rights | Immutable; wider scope means a new job and a new quote. |
| Ticket | One billable generation job | Owns user-visible identity and the billing subject. |
| Assignment | One provider attempt | A retry creates a new attempt, not a new ticket. |
| Executor | Provider adapter / GPU worker | Isolated workspace; reports facts, does not write job state directly. |
| Subscription | Project/conversation context | Context only; no expansion of authority. |
| Outbox | Charge, refund, webhook, email, asset publish | Record first, execute after commit; ordered per ticket. |
| Policy | Model catalog + price table + safety rules | Versioned; every quote and mandate records the version. |
| Watchdog | Stuck job, overbudget, loop, provider incident | May stop, downgrade or release; never expand budget. |
Turning the conclusion into a skill
The SKILL.md below is not DeepSeek’s text. It is my translation of the prototype’s reusable control plane into media generation. It gives a coding agent constraints rather than another role: authorize before spending, one request one ticket, context does not expand scope, retries do not fork the bill, record before the side effect, reconcile before retry.
To use it, copy it to ~/.codex/skills/media-generation-harness/SKILL.md, or put it in a project’s .agents/skills/. It triggers on generation jobs, provider adapters, credits and billing, retries and reconciliation, queues, moderation and asset delivery.
Open the full SKILL.md (copyable)
---
name: media-generation-harness
description: Design, build, or audit a durable backend for an AI image/video generation product. Use when the task involves generation jobs, provider adapters, credits and billing, retries and reconciliation, queues, moderation gates, asset delivery, or turning a prototype into a production media site.
---
# Media Generation Harness
Use this skill when building or reviewing the control plane behind an AI image
or video product. The goal is not "an agent that can call a model". The goal is
a system that spends as little user attention and money as possible per
delivered asset, and can prove what it did after a crash.
## When to use
- A generation request must survive restarts, provider timeouts and duplicate
webhooks.
- Users pay credits, so authorization, reservation, settlement and refund must
be auditable.
- More than one provider or model will be tried behind one product surface.
- The UI needs progress and failure states without streaming raw logs.
- A prototype works locally but cannot safely take production traffic.
Do not use this skill to design prompt content, model quality benchmarks or
creative direction. Those are separate jobs.
## Non-negotiable invariants
1. **Authorize before spend.** No provider call happens before ownership,
quota, price and policy checks pass.
2. **One immutable request, one ticket.** A client retry must resolve to the
same job, not a second charge.
3. **Deterministic gate before model judgement.** Auth, credits, catalog,
resolution, duration, rate limit and pre-moderation are code, not prompts.
4. **Context never expands scope.** A later chat message may add creative
context, but it cannot raise budget, resolution, duration or provider scope.
Scope changes need a new mandate.
5. **One ticket, many attempts.** A provider retry creates a new attempt under
the same job. Never fork billing or user-visible identity on retry.
6. **Persist before effect.** Reserve credits, write the outbox row and record
the provider request before performing the external action.
7. **Reconcile before retry.** If provider state is unknown, query it. Never
resubmit a job that may already be running.
8. **Exception-only UI.** Show "needs input", "ready to review" and "failed
with one action". Healthy work is quiet.
9. **The watchdog may stop or downgrade, never expand.** It cannot increase
budget, resolution, duration or permissions.
10. **Price, catalog and safety policy are versioned.** Every quote and mandate
records the version it relied on.
## Planes
| Plane | Responsibility | Must not do |
| --- | --- | --- |
| Control | intake, quote, gate, authorize, schedule, settle, notify | run GPU work |
| Execution | submit to provider, poll, download, validate, post-process | decide price or permission |
| Data | jobs, attempts, ledger, assets, events, policy versions | hide side effects in memory |
Treat the control plane as the only writer of job state. Workers report facts;
they do not mutate the database directly.
## Domain model
| Concept | Media-site meaning | Rule |
| --- | --- | --- |
| Source | Web upload, API, Stripe, moderation callback | Normalize and dedupe by source event id |
| Trigger | One immutable request or callback | Stored before processing |
| Thread | Project or conversation | Holds context; does not grant budget |
| Gate | Auth, credits, quota, catalog, cost ceiling, pre-moderation | Deterministic and synchronous |
| Assessment | Prompt clarity, reference suitability, policy risk | Advisory; cannot set price or permission |
| Decision | accept, clarify, reject, defer | `clarify` asks exactly one question |
| Mandate | Model, resolution, duration, max cost, fallback list, rights | Immutable; wider scope means a new mandate |
| Ticket | One billable generation job | Owns user-visible identity and ledger |
| Assignment | One provider attempt | Retries create a new assignment |
| Executor | Provider adapter or GPU worker | Isolated workspace and asset prefix |
| Subscription | Project/thread context | Context only; no new authority |
| Channel | SSE/WebSocket, webhook, email | Ordered per ticket |
| Outbox | Charge, refund, webhook, email, publish | Record first, send after commit |
| Policy | Model catalog, price table, safety rules | Versioned |
| Watchdog | Stuck, overbudget and loop detection | External control loop |
## Job state machine
```text
draft
-> quoted
-> authorized
-> queued
-> submitted
-> running
-> downloaded
-> validated
-> ready
-> delivered
any active state -> needs_input
any active state -> blocked
any active state -> failed
any active state -> canceled
quoted/authorized -> expired
```
Rules:
- `authorized` is the only state from which a provider may be called.
- `submitted` and `running` must carry a provider job id and the exact request
payload hash.
- `failed` must carry a normalized error code, not only a provider string.
- `canceled` must record whether the provider acknowledged cancellation.
- Terminal states are immutable except for append-only audit events.
## Money path
1. Quote from the current price policy version.
2. Reserve credits with an idempotency key derived from the ticket.
3. Commit the job and outbox row in one transaction.
4. Submit to the provider only after commit.
5. On definite submit failure, release the reserve.
6. On success, settle the actual amount; release the difference.
7. On failure, apply the published refund rule, not an ad-hoc decision.
8. Reconcile provider usage and ledger daily.
Never charge from a webhook handler without checking the stored job and
attempt. Webhooks are delivery mechanisms, not sources of truth.
## Provider adapter contract
```ts
export interface ProviderAdapter {
readonly id: string
quote(input: NormalizedRequest, policy: PolicyVersion): Promise<Quote>
submit(input: {
request: NormalizedRequest
attemptId: string
idempotencyKey: string
callbackUrl: string
}): Promise<{
providerJobId: string
acceptedAt: string
raw: unknown
}>
get(providerJobId: string): Promise<{
state: "queued" | "running" | "succeeded" | "failed" | "canceled" | "unknown"
progress?: number
output?: OutputRef[]
error?: { code: string; message: string; retryable: boolean }
raw: unknown
}>
cancel(providerJobId: string): Promise<{
accepted: boolean
state: "canceled" | "running" | "unknown"
}>
parseWebhook(headers: Headers, body: unknown): Promise<{
providerJobId: string
eventId: string
state: string
raw: unknown
}>
}
```
Adapters must normalize provider states, preserve raw payloads for audit and
never expose provider URLs as permanent assets.
## Minimum data model
- `generation_requests`: client idempotency key, owner, raw input hash.
- `quotes`: policy version, line items, currency, expiry.
- `jobs`: ticket id, mandate, state, current attempt id.
- `attempts`: provider, provider job id, request hash, response, timings.
- `credit_ledger`: append-only reserve, settle, release, refund entries.
- `outbox`: unique `(ticket_id, effect_type, effect_key)`, attempts, sent time.
- `provider_events`: unique `(provider, event_id)`, raw payload, processed time.
- `assets`: storage key, checksum, dimensions, duration, moderation result.
- `policy_versions`: catalog, price table, safety rules, effective time.
Use unique indexes for idempotency. A retry that violates one should return the
existing record, not an error the client retries again.
## Gates
Run these in order and stop at the first failure:
1. Authentication and ownership.
2. Request shape and size.
3. Model, resolution and duration allowed by the catalog.
4. Credit balance and per-user/per-IP quota.
5. Cost ceiling for this request.
6. Pre-moderation on prompt and reference assets.
7. Provider availability and fallback eligibility.
Return a stable error code and one actionable next step. Do not return a wall
of provider text.
## Assessment
Use a model only for genuinely semantic questions:
- Is the prompt specific enough to generate the requested asset?
- Is the reference image suitable for the requested edit?
- Does the request appear to violate a policy that the deterministic filter
cannot express?
The assessment returns `accept`, `clarify` or `reject`, with evidence. It never
sets price, credits, resolution or provider permission.
## Recovery
On startup:
1. Find jobs in `submitted` or `running`.
2. Query the provider for each known provider job id.
3. If the provider has a result, continue from that result.
4. If the provider has no record and the attempt is past its submit deadline,
mark the attempt failed and allow a new attempt.
5. Never resubmit solely because the local process restarted.
6. Release or settle reserves only after reconciliation.
Store enough provider request context to explain what was sent, but never store
provider credentials in job payloads or prompts.
## Watchdog
Start with deterministic checks; add a model only when the checks cannot
express the failure.
| Check | Action |
| --- | --- |
| Job older than p99 for its model | query provider, then cancel or escalate |
| No provider state change for N minutes | query, then retry with fallback |
| Cost exceeds mandate | stop, release reserve, alert |
| Same error three times | open a provider incident, stop retrying |
| Queue wait exceeds SLO | add capacity or degrade catalog |
| Duplicate webhook storm | dedupe, alert, do not create jobs |
The watchdog has a read-only view of policy and a narrow write set: cancel,
retry within the existing mandate, release reserve, open incident. It cannot
raise budget or permissions.
## Human attention budget
Expose only four user-facing states:
- **Working**: progress without logs.
- **Needs you**: one question, one input.
- **Ready**: preview, download, accept or regenerate.
- **Failed**: one plain-language reason and one next action.
Do not stream chain-of-thought, provider polling or worker logs. Put them
behind a developer trace view.
## Security
- Treat prompts, uploaded files, reference URLs and provider responses as
untrusted input.
- Never let user content change policy, price, budget or provider credentials.
- Keep provider keys server-side; never place them in model context.
- Use signed, expiring asset URLs; mirror outputs to your own storage.
- Moderate input and output; record the moderation policy version.
- Add rate limits, abuse detection and a manual kill switch.
- Consider watermarking and content provenance for generated media.
## Speculative work
Speculation is only safe when the marginal cost is small and the result can be
discarded silently. For media:
- Safe: prompt cleanup, safety pre-check, cost estimate, thumbnail, 480p draft.
- Unsafe by default: full 1080p/4K render, long video, paid third-party call
with no cap.
Cap speculative spend per request and per day. Never let speculation create a
user-visible charge.
## Build order
1. Schema, idempotency keys and append-only ledger.
2. Quote and deterministic gate.
3. One provider adapter and one worker.
4. Outbox and crash-safe submit.
5. Polling plus webhook reconciliation.
6. Asset mirroring, checksum and validation.
7. Exception-only UI.
8. Watchdog and incident view.
9. Second provider and fallback policy.
10. Shadow routing, then percentage rollout.
## Tests that must exist
- Same idempotency key twice returns the same job and one charge.
- Crash after reserve but before submit releases or reconciles correctly.
- Provider timeout followed by success does not double-submit.
- Duplicate webhook creates one event and one settlement.
- Cancel racing with success produces one terminal state and one ledger result.
- Restart during `running` reconciles before any retry.
- Overbudget request is rejected before the provider call.
- Prompt injection cannot change price, budget or provider.
- Provider URL expiry is handled by the mirrored asset.
- Watchdog cannot exceed the mandate.
## Anti-patterns
- An LLM deciding price, credits or permission.
- Charging after submit, or charging from a webhook alone.
- Retrying an unknown provider state.
- Treating a provider URL as permanent storage.
- One mutable `status` string for the whole lifecycle.
- Streaming logs to the main product UI.
- Multi-agent role-play where a deterministic queue would do.
- Speculative full-resolution renders with no cap.
- Storing provider keys in prompts, job payloads or logs.
- Adding a second provider before the first one is reconcilable.
## Output checklist
- [ ] Every external effect has an outbox row and an idempotency key.
- [ ] Every job has an immutable mandate and a policy version.
- [ ] Every attempt has a provider job id, request hash and normalized result.
- [ ] Credits reserve, settle and release through the ledger only.
- [ ] Restart and duplicate-webhook tests pass.
- [ ] The UI has exactly four user-facing states.
- [ ] The watchdog can stop and downgrade, but not expand.
- [ ] Provider credentials never enter model context.
Turning the constraints into a harness contract
harness.yaml is not code. It is a reviewable contract: planes, concepts, job states, gates, the money path, the provider adapter, recovery, watchdog, UI, security and SLOs in a machine-readable shape. Put it in the repository and use it as a PR checklist, or let a coding agent generate the schema, state machine and test list from it.
Open the full harness.yaml (copyable)
version: 1
name: media-generation-harness
status: reference-contract
principle: "Spend as little user attention and money as possible per delivered asset."
planes:
control:
owns:
- intake
- quote
- gate
- authorization
- scheduling
- settlement
- notification
must_not:
- run_gpu_work
- trust_provider_text_as_state
execution:
owns:
- provider_submit
- provider_poll
- download
- validate
- post_process
must_not:
- set_price
- grant_permission
- write_job_state_directly
data:
owns:
- jobs
- attempts
- credit_ledger
- assets
- provider_events
- policy_versions
must_not:
- hold_side_effects_only_in_memory
concepts:
source:
examples:
- web_upload
- public_api
- stripe
- moderation_callback
rule: "Normalize the event and dedupe by source event id before it enters the domain."
trigger:
rule: "Immutable request or callback, stored before processing."
thread:
rule: "Project or conversation context. Never grants budget or permission."
gate:
rule: "Deterministic checks only. Runs before any model assessment."
assessment:
rule: "Semantic advice only. Cannot set price, credits, resolution or provider."
decision:
values:
- accept
- clarify
- reject
- defer
rule: "clarify asks exactly one concrete question."
mandate:
fields:
- policy_version
- model
- resolution
- duration_seconds
- max_cost
- fallback_providers
- rights
- accepted_by
rule: "Immutable. Any wider scope requires a new ticket and mandate."
ticket:
rule: "One billable generation job. Owns user-visible identity and ledger."
assignment:
rule: "One provider attempt. Retries create a new assignment under the same ticket."
executor:
rule: "Provider adapter or GPU worker with an isolated workspace and asset prefix."
subscription:
rule: "Context only. It never expands authority."
channel:
examples:
- sse
- websocket
- webhook
- email
rule: "Ordered per ticket."
outbox:
rule: "Record the effect before performing it. Unique per ticket and effect key."
policy:
rule: "Catalog, price table and safety rules are versioned together."
watchdog:
rule: "External control loop. May stop or downgrade, never expand."
job:
states:
- draft
- quoted
- authorized
- queued
- submitted
- running
- downloaded
- validated
- ready
- delivered
- needs_input
- blocked
- failed
- canceled
- expired
transitions:
- from: draft
to: quoted
guard: "catalog entry exists and quote is complete"
- from: quoted
to: authorized
guard: "gate passed and credits reserved"
- from: authorized
to: queued
guard: "outbox row committed"
- from: queued
to: submitted
guard: "provider accepted the request"
- from: submitted
to: running
guard: "provider reports running"
- from: running
to: downloaded
guard: "output mirrored to owned storage"
- from: downloaded
to: validated
guard: "checksum, dimensions and moderation passed"
- from: validated
to: ready
guard: "asset manifest committed"
- from: ready
to: delivered
guard: "channel delivery recorded"
- from: active
to: needs_input
guard: "one structured question is pending"
- from: active
to: blocked
guard: "external dependency cannot proceed"
- from: active
to: failed
guard: "normalized terminal error recorded"
- from: active
to: canceled
guard: "cancel requested and provider state reconciled"
- from: quoted
to: expired
guard: "quote TTL elapsed"
- from: authorized
to: expired
guard: "authorization TTL elapsed before submit"
terminal:
- delivered
- failed
- canceled
- expired
rules:
- "Only authorized may call a provider."
- "submitted and running must carry provider_job_id and request_hash."
- "Terminal states are immutable except for append-only audit events."
gates:
order:
- id: auth
checks: ["session", "api_key", "ownership"]
- id: shape
checks: ["schema", "file_size", "url_allowlist"]
- id: catalog
checks: ["model", "resolution", "duration", "aspect_ratio"]
- id: quota
checks: ["credit_balance", "user_rate", "ip_rate"]
- id: cost
checks: ["max_cost", "daily_budget", "provider_price_version"]
- id: pre_moderation
checks: ["prompt", "reference_assets"]
- id: availability
checks: ["provider_health", "fallback_eligibility"]
failure_contract:
return:
- stable_error_code
- one_actionable_next_step
- policy_version
never:
- raw_provider_dump
- ambiguous_retry_instruction
assessment:
enabled: optional
inputs:
- normalized_prompt
- reference_asset_metadata
- policy_version
outputs:
- accept
- clarify
- reject
evidence_required: true
cannot:
- set_price
- set_credits
- raise_resolution
- extend_duration
- grant_provider_permission
money:
ledger: append_only
entries:
- reserve
- settle
- release
- refund
flow:
- "quote from policy version"
- "reserve with ticket-derived idempotency key"
- "commit job and outbox in one transaction"
- "submit after commit"
- "release on definite submit failure"
- "settle actual on success"
- "apply published refund rule on failure"
- "reconcile provider usage daily"
forbidden:
- "charge from webhook without stored job and attempt"
- "charge after submit with no reserve"
- "ad_hoc_refund"
provider_adapter:
methods:
quote:
input: ["normalized_request", "policy_version"]
output: ["line_items", "currency", "expiry"]
submit:
input: ["normalized_request", "attempt_id", "idempotency_key", "callback_url"]
output: ["provider_job_id", "accepted_at", "raw"]
get:
input: ["provider_job_id"]
output: ["state", "progress", "output", "error", "raw"]
cancel:
input: ["provider_job_id"]
output: ["accepted", "state"]
parse_webhook:
input: ["headers", "body"]
output: ["provider_job_id", "event_id", "state", "raw"]
normalized_states:
- queued
- running
- succeeded
- failed
- canceled
- unknown
rules:
- "Preserve raw provider payloads for audit."
- "Never expose provider URLs as permanent assets."
- "Never resubmit when provider state is unknown."
data_model:
generation_requests:
keys: ["client_idempotency_key", "owner_id"]
fields: ["raw_input_hash", "created_at"]
quotes:
keys: ["request_id", "policy_version"]
fields: ["line_items", "currency", "expires_at"]
jobs:
keys: ["ticket_id"]
fields: ["mandate", "state", "current_attempt_id"]
attempts:
keys: ["attempt_id"]
fields: ["provider", "provider_job_id", "request_hash", "response", "timings"]
credit_ledger:
keys: ["entry_id"]
fields: ["ticket_id", "kind", "amount", "policy_version", "created_at"]
outbox:
keys: ["ticket_id", "effect_type", "effect_key"]
fields: ["payload", "attempts", "sent_at", "last_error"]
provider_events:
keys: ["provider", "event_id"]
fields: ["raw", "processed_at"]
assets:
keys: ["asset_id"]
fields: ["storage_key", "checksum", "width", "height", "duration", "moderation"]
policy_versions:
keys: ["policy_version"]
fields: ["catalog", "price_table", "safety_rules", "effective_at"]
recovery:
startup:
- "find jobs in submitted or running"
- "query provider for each known provider_job_id"
- "continue from provider result when present"
- "fail attempt when provider has no record and submit deadline passed"
- "never resubmit only because the local process restarted"
- "settle or release only after reconciliation"
invariants:
- "Unknown provider state blocks retry."
- "Duplicate webhook is a no-op after the first processed event."
- "Cancel racing with success yields one terminal state and one ledger result."
watchdog:
checks:
- id: stale_job
condition: "age > p99 for model"
action: ["query_provider", "cancel_or_escalate"]
- id: no_progress
condition: "no provider state change for N minutes"
action: ["query_provider", "retry_with_fallback"]
- id: overbudget
condition: "cost > mandate.max_cost"
action: ["stop", "release_reserve", "alert"]
- id: repeated_error
condition: "same normalized error three times"
action: ["open_incident", "stop_retrying"]
- id: queue_slo
condition: "queue wait > SLO"
action: ["add_capacity", "degrade_catalog"]
- id: webhook_storm
condition: "duplicate event rate above threshold"
action: ["dedupe", "alert"]
write_set:
- cancel
- retry_within_existing_mandate
- release_reserve
- open_incident
cannot:
- raise_budget
- raise_resolution
- extend_duration
- grant_permission
ui:
user_states:
working:
shows: ["progress"]
hides: ["logs", "chain_of_thought", "provider_polling"]
needs_input:
shows: ["one_question", "one_input"]
ready:
shows: ["preview", "download", "accept", "regenerate"]
failed:
shows: ["plain_reason", "one_next_action"]
developer_trace:
includes: ["provider_request_id", "normalized_error", "timings", "policy_version"]
security:
untrusted:
- prompt
- uploaded_file
- reference_url
- provider_response
rules:
- "User content cannot change policy, price, budget or credentials."
- "Provider keys stay server-side and never enter model context."
- "Assets use signed expiring URLs and are mirrored to owned storage."
- "Moderate input and output; record the moderation policy version."
- "Rate limit, detect abuse and provide a manual kill switch."
- "Consider watermarking and content provenance."
speculation:
allowed:
- prompt_cleanup
- safety_precheck
- cost_estimate
- thumbnail
- draft_480p
forbidden_by_default:
- full_1080p_render
- full_4k_render
- long_video
- uncapped_third_party_call
rules:
- "Cap speculative spend per request and per day."
- "Never create a user-visible charge for speculative work."
- "Discard silently when the result is not useful."
slo:
track:
- quote_latency
- queue_wait
- submit_latency
- provider_duration
- success_rate
- cost_per_delivered_asset
- duplicate_charge_rate
- reconciliation_lag
targets:
duplicate_charge_rate: 0
ledger_mismatch_rate: 0
reconciliation_lag_minutes: 15
Adoption order and anti-patterns
If you have a prototype today, build in this order instead of starting with a “multi-agent team”:
- 01Schema, idempotency keys and an append-only credit ledger.
- 02Quote and deterministic gate. Do not connect a provider before these exist.
- 03One provider, one worker; get submit / poll / cancel working.
- 04Outbox and crash-safe submit; prove that a restart does not double-submit.
- 05Webhook plus polling reconciliation; a duplicate webhook must be a no-op.
- 06Asset mirroring, checksum, dimensions/duration validation and moderation.
- 07Four-state UI and an exception queue. Do not start with a log stream.
- 08Deterministic watchdog; only then consider a model for “is it drifting?”.
- 09A second provider, fallback and shadow routing.
What not to copy
- All eleven concepts. A two-person media site needs job, attempt, ledger and outbox first. Do not start with Correlation, Subscription and the full Policy model.
- The plugin architecture. “Everything is a plugin” is an advantage for a harness substrate, not an MVP requirement for your site.
- Multi-agent role-play. One queue, a pool of workers and an exception panel is usually more reliable than “Router Agent, Planner Agent, Critic Agent”.
- An LLM watchdog. Make stuck jobs, overbudget and repeated errors deterministic first. Use a model only for “it is progressing but in the wrong direction”.
- Speculative full renders. Video marginal cost is too high; speculate only on cheap upstream steps.
- Provider URLs as storage. They expire, rate-limit and disappear. Mirror the bytes.
- Five providers before reconciliation. If one provider’s reconciliation is not working, five will only multiply the accounting errors.
If you remember one sentence: the thing worth copying from DeepSeek is not multi-agent role-play. It is turning “this work may be done” into an auditable mandate, then hanging execution, retry, recovery and notification underneath it. For an AI video/image site, that means quote / gate / ledger first, provider scheduling second; no duplicate charges and no duplicate submissions before autonomy.
Sources and verification
All links returned 200 on 2026-11-02. The commit, file counts and documentation conclusions come from the local clone. The experiment numbers come from the Chinese full-text translation of the design essay; I do not treat them as an independent benchmark.
- DeepSeek Harness · Product pageOfficial product page: Cordis plugin architecture, desktop and web, MIT
- DeepSeek Harness · QuickstartDeveloper documentation and installation path
- deepseek-ai/deepseek-harnessThe open-source harness repository
- deepseek-ai/dsh-superagent-prototypeResearch prototype inspected at commit 6337fc8; README, docs/ and src/
- Machine Heart · Super-agent design (Chinese translation)Full Chinese translation of the DeepSeek Harness team essay; used because the original WeChat post is not stably fetchable
- GitHub · dsh-plugin topicCommunity plugin ecosystem referenced by the product page
If DeepSeek updates the prototype, re-check three places first: the README disclaimer, docs/001-domain.md through 003-execution.md for concepts and state machines, and prompts/ for admission and worker constraints. The essay-vs-code table should be updated with them.