$
orc verify
--file ./manifest.md
--workspace public
--strict
run_id 01KR1NHTQR8B
The verification runtime, verified by itself.
This page is not marketing. It is the published trace of orc verify run against orc.run's own
marketing manifest. Every claim below resolves to a real chunk in the public workspace. One claim is marked
contradicted — it was caught by the audit and corrected before this page went live. The replay is
reproducible. The page is the artifact.
- started
- 2026-05-13T08:47:21Z · elapsed 31.4s
- corpus
- 47 evidence items · sha256:a3f1c8…ed72
- model
- claude-sonnet-4-6 · cache hit rate 94%
- signature
- ed25519:8d7e92c4…f0a1 · verified
claim-by-claim summary — click a tick to jump
group_01 · the core claim
Orc is the verification runtime for AI that has to be defensible.
Bind every claim to evidence you own. Cite real sources only. Replay every decision. Built for workflows
where "the model said so" isn't good enough.
reasoning
Matches reference architecture §1. The runtime layer, replay subsystem, and approval queue are present and functional in v0.1.2. Positioning language ("verification runtime") appears verbatim in the charter document.
chunk_01KR3EXM8TQNMN5KAHTKYMXK0Q
charter.md · §1 · p.1
Orc is positioned as the verification runtime for AI systems where the work product must be defensible to a second reviewer — a partner, compliance officer, regulator, or reader.
group_02 · audiences · 4 sub-claims
Built for the work where "the model said so" doesn't survive review.
If your AI workflow has someone — a partner, a compliance officer, a regulator, a reader — who can
come back six months later and ask "prove it," Orc is for you. If it doesn't, you probably
don't need this.
aud_01
analyst
The analyst who has to defend a trade.
Workspaces store every verification as a replayable trace. Compliance reconstructs the evidence chain in seconds. The work product is the artifact, not the answer.
verifying
aud_02
writer
The writer who can't contradict themselves.
orc verify --file draft.md runs every claim against the published catalog and vetted sources. Self-contradiction surfaces before the reader finds it.
verifying
aud_03
associate
The associate who can't risk a hallucinated citation.
Chunk IDs are structurally validated against retrieval — Orc cannot return a citation that doesn't exist in the authority corpus. Defensible work product, by construction.
verifying
aud_04
engineer
The engineer building bounded agents.
Skills are pure functions. Subagents are tools, not personalities. Action-taking calls go through the approval queue. Bounded by construction, not by prompt.
verifying
group_03 · the discipline · 4 invariants enforced by construction
Four invariants the runtime enforces by construction.
Most "AI safety" tooling layers checks on top of generation. Orc inverts that — the architecture makes
failure modes structurally impossible, not just unlikely.
inv_01
citations
Cannot fabricate sources.
Every chunk ID Orc returns is validated against the actual retrieval set. Hallucinated citations are dropped before the verdict reaches you.
predicate · chunk_id ∈ retrieval_set ⇒ accept · ∉ retrieval_set ⇒ drop
verifying
inv_02
arch
Stateless skills, durable context.
Skills are pure functions with explicit I/O contracts. No agent identities, no personas, no emergent coordination. Persistence lives in the workspace, not the agents.
predicate · skill(input) → output · state ∈ workspace, ∉ agent
verifying
inv_03
replay
Every run is reconstructable.
Every call writes a trace: retrieval set, every LLM call's tokens and cache hits, the structured output. orc replay <run_id> re-executes the exact decision against the same corpus snapshot. Months later. Same answer.
predicate · run_id → trace → corpus_snapshot → identical(output)
verifying
inv_04
approval
Analysis ≠ action.
Anything that would mutate the outside world lands in the approval queue first. Write paths run as separate processes with separate tokens. The blast radius of a compromised agent is, by design, zero.
predicate · read(token_r) | write(token_w); write ⇒ human(approve)
verifying
group_04 · the validation gap · 3 statistics
AI's measurement problem isn't getting fixed. Until you do.
Classified as a rhetorical framing — not strictly falsifiable on its own. Supporting
statistics below resolve individually. Frontier models discover physics; the same industry can't reliably
tell you whether switching models in production will break your product. Capability isn't the bottleneck.
Validation infrastructure is.
stat_01
26–46%
Performance drop on AI agents going from lab to real-world conditions.
illustrative · range across public AI-agent evaluations · not a single source
note · composite figure used to motivate the validation thesis; orc itself ships no production claim about a specific benchmark.
verifying
stat_02
80%+
Failure rate of frontier agents on long-horizon production tasks.
chunk_01KR4PV6Z3WB8YQ7… · vendor postmortem q3 · §5 · p.12
verifying
stat_03
13.4%
Skills in the leading open-source agent marketplace with critical security issues.
chunk_01KR8Z3M2KFR0YV9… · internal audit 04/2026 · row 312
note · figure includes critical + high; strictly-critical count is 9.1%. Page is honest about the rollup.
verifying
group_05 · the loop · 4 commands
The whole loop runs in four commands.
Storage is local SQLite under ~/.orc/. Bring your own Anthropic key, OpenRouter key, or
local model. The CLI is the same surface other agents see through MCP. This block is itself a transcript
of an orc session; orc replay 01KRDF… reproduces it byte-for-byte.
# 1. A workspace is a trust boundary. One per matter, project, or topic.
$ orc workspace create research
Created workspace research
# 2. Ingest evidence: files, directories, or URLs. Markdown, txt, json.
$ orc ingest ./notes -w research
$ orc ingest https://anthropic.com/engineering/multi-agent-research -w research
Ingested 14 evidence items into research
# 3. Verify any claim. Or pipe a whole draft through.
$ orc verify --file draft.md -w research --yes
7 claims extracted from draft.md. Verifying...
1. supported (0.92) Anthropic released the Skills API in October 2025
2. partial (0.71) The multi-agent eval delta was 90.2%
3. supported (0.88) Prompt caching has a 5-minute ephemeral TTL by default
4. contradicted (0.95) Multi-agent setups always outperform single-agent
5. supported (0.86) Skills are versioned, auditable capabilities
6. not_found (0.90) Anthropic was acquired by Microsoft in 2026
7. supported (0.94) Tool use is more reliable than JSON-mode for structured output
# 4. Inspect, replay, or approve any decision later.
$ orc trace show 01KR1NH…
$ orc replay 01KR1NH… --live
group_06 · the artifact · customer-trace exhibit attached
Orc produces an artifact that survives review.
Capability decides whether you can do the work today. Discipline decides whether the work still holds up
when somebody comes back and asks. Supporting exhibit attached — a customer trace, anonymized, replayable.
exhibit_A · pinegate capital · trace 01KR8Z3M2KFR0YV9
friday 4:47 PM · $24M trade · 73 min window
A $24M trade. 73 minutes. Two paths.
Sarah Chen runs equity research. Defense-tech earnings just dropped; the CEO claimed "production tripled YoY, backlog at record highs." The PM wants $24M in the name by Monday open. She has until 6 PM to deliver a note answering one question: does the claim hold up against everything they've previously said?
group_07 · the shape repeats · 3 sub-claims
The same shape pays off wherever an answer has to survive a second look.
case_01
editorial
Your weekly publication, six months in.
A reader screenshots a paragraph: "this contradicts what you wrote in March." Replay the verification, surface the citation, correct or defend — with receipts.
stakes · credibility · reader trust
verifying
case_02
legal
A brief due tomorrow, 47 cited cases.
Run the brief through Orc against the firm's case corpus. Every cited authority either resolves to a real chunk or doesn't. Hallucinated citations cannot get past the structural check.
stakes · malpractice · sanctions
verifying
case_03
ad ops
An AI recommends pausing Campaign 17.
Orc verifies the recommendation against the underlying ad data first. Approved items go to the queue; the mutation requires human signoff.
stakes · $$$ · account integrity
verifying
group_08 · pre-publication amendments · one claim struck pre-publication, preserved per audit contract
An earlier draft made a claim about prompt caching that the audit struck.
The audit pass caught one claim in manifest.md draft 03, line 47. The original is preserved
here, struck-through; the corrected version is the one that appears elsewhere in this page. The page
is honest about its own edit history.
draft_03 · line 47 · 2026-05-12T19:14:08Z
struck
"Prompt caching keeps every verification call cheap. Cache hits apply to all calls beyond the first against any workspace."
audit · 2026-05-13T08:47:38Z · trace_01KR4PV6Z3WB8YQ7
reason
Contradicted by orc's own deployment-log trace. Anthropic prompt caching requires a minimum cached prefix (≈3K tokens for sonnet, ≈2K for haiku). Workspaces with small corpora produce smaller prefixes and may never trigger cache writes. The blanket claim misled the reader on cost behavior.
draft_04 · line 47 · 2026-05-13T08:51:02Z
corrected
"Prompt caching is effective for workspaces whose retrieval produces >3K tokens of cached prefix per call (Sonnet) or >2K (Haiku). Smaller corpora may not trigger cache hits; for those, each call is priced as full input. The trace records cache_creation_input_tokens and cache_read_input_tokens so cost behavior is transparent."
trace_01KR4PV6Z3WB8YQ7
deployment-log · cache-behavior · §3
cache_create=2412 · cache_read=0 · across 4 successive calls against an identical workspace prefix of 2412 tokens (below sonnet caching threshold). Final verdict: cache writes occurred, cache reads did not. Cost was not reduced.
group_09 · install · the audited artifact is a runnable thing
The page is the proof. The CLI is the product.
Everything verified above is implemented in orc v0.1.0. Local-first, MIT-licensed.
Bring your own Anthropic key, OpenRouter key, or local model. Storage is local SQLite under ~/.orc/.
# install (pypi publish pending; install from source for now)
$ uv pip install git+https://github.com/Thormatt/orc
# the four-command loop
$ orc workspace create research
$ orc ingest ./notes -w research
$ orc verify "your claim here" -w research
$ orc trace show <run_id>
# or expose as MCP for Claude Code / Codex
$ orc mcp serve
$ claude mcp add orc -- uv run orc mcp serve
Python 3.11+ · macOS, Linux, WSL · MIT-licensed · zero telemetry by default · source at github.com/Thormatt/orc