# how to build a harness

> the models got great and the chatbots are wasting them. the harness is the part you own, and i own two.

2026-08-23 -- 17 min -- ai, engineering, tools

canonical: https://rohans.lol/blog/how-to-build-a-harness

---
fable 5 can drive a computer for six hours. i know because [i watch it happen](/blog/box-box), half the time from my phone: threads that run all day and spawn side threads with agents i never talk to directly. the biggest single day was 82 commits, twenty-three thousand lines, and i remember it mostly as a conversation. that's the machine. now guess how most of the world uses it.

a chat window. you type, it types back. that's the entire relationship. the six hour machine, reduced to autocomplete with manners.

the models got so good that the products wrapping them are now the bottleneck, and nobody seems to think this is insane. chatgpt is a brilliant brain in a padded room. claude.ai is the same brain in the padded room across the hall. the rooms gain features every quarter, and the features never add up to a job. memory remembers who you are and how you like your answers. it does not remember work. the sandbox runs real python and evaporates when the conversation ends. no repo, no working state the model keeps, no computer that's yours. supervised visits, not employment.

and you can see the ceiling in the usage data. openai ran [the largest study of chatgpt use ever published](https://www.nber.org/papers/w34255): nearly 80% of conversations are three things, practical guidance, looking stuff up, and writing help. two thirds of the writing isn't even writing, it's people pasting their own paragraph in and asking for it back, nicer. programming is 4%. the same class of model that runs my repos all day is spending its public life rewording emails. and that's not a billion people lacking imagination. a text box teaches you the size of an ask. give people an upload button and they will send you a document. give them a computer and they will hand you a job.

chatbots are how you interview a model. a harness is how you employ it.

this post is me taking the panels off my two: [codemaxxxing](/blog/codemaxxxing-isnt-just-a-fork-anymore), the harness i live in, and clauseo, the one my lawyers use, which had to reinvent what the word computer even means. one lesson, two machines, opposite ends of the same story. that's the post.

## the api is a brain in a jar

anthropic people keep describing it as brain and hands. the model is the brain, everything around it is the hands. i'd go further. what a frontier lab actually sells you is a brain in a jar. the entire api is this: you send a list of messages, you get one message back. that is the whole product. no files, no shell, no memory, no computer, no idea what day it is unless you tell it. the labs are bolting hands onto the jar as fast as they can, server side code execution, memory apis, computer use. the bare endpoint is still what everything else is built on.

every impressive thing you have ever seen an ai agent do, every file edited, every test run, every browser driven, every pr opened, was not the jar. it was plumbing somebody built around the jar. the plumbing is the harness, and every harness ever built is answering the same six questions:

**the loop.** when the brain asks to run something, who runs it, and how many times do you go around before stopping.

**the window.** what exactly you put in front of the brain each turn, because that array is its entire perceived reality.

**the hands.** which tools it gets, and what those tools are allowed to drag back into the window.

**the state.** what survives between turns, sessions, crashes.

**the leash.** what it can do without asking a human first.

**the bill.** caching and routing, the meter that decides whether any of this is economically sane.

same brain, different answers, completely different animal. it's why the same model feels like a staff engineer in one product and a goldfish in another, and why the labs publish agentic benchmarks with a quiet footnote that says "with our scaffold". the scaffold is doing a stupid amount of the work.

and it's why i keep saying the model is not your product. everyone rents the same four brains. you, me, every billion dollar legal ai company. the harness is the part you own.

## the loop is a while loop, calm down

strip everything away and an agent is this: assemble context, call the model, if it asked for tools then run them, append the results, go again. when it stops asking, you're done.

that's it. that's the mystical agentic loop. you can write it in an afternoon, and you should, it's the most educational afternoon available in software right now. i've [written before](/blog/codemaxxxing-context-engineering) about what happens when you run one overnight in a loop and let the files carry the memory.

but the loop is not the product. the loop is the chassis number. the product is the several hundred decisions bolted around it, and the rest of this post walks the ones that hurt, roughly in the order they'll hurt you.

## the model's entire reality is one array

every turn, the harness rebuilds the model's world from nothing. system prompt, tool schemas, instruction files, the conversation so far, every tool result. one array. the model knows your codebase exists because the harness said so this turn. it knows the tests failed because the harness pasted the failure in. there is no other input channel. no ambient awareness, no peripheral vision. if the harness didn't put it in the array, it does not exist.

once that clicks you stop treating instruction files as config. they are the highest leverage files you will ever write. codemaxxxing reads AGENTS.md files at every level, global for the machine, per-repo for the project, and the layering is the point: keep each level small, let the model pull detail when it needs it instead of carrying everything everywhere. because the brain in the jar is brilliant and amnesiac, and the only cure for amnesia anyone has ever invented is writing things down where you'll be forced to reread them. the files remember so the model doesn't have to. when the model fumbles, the fix goes in the file, and the harness shoves the file back into reality every session forever. make the harness better, every time it fumbles, in a file it rereads.

## the window is a lie

and then the array gets long. it always gets long, and the number on the model card will betray you.

every model ships with a number. one million tokens. 500k. the number is technically true and practically a lie, because the model you get at 400k is not the model you paid for at 40k. [nolima measured this properly](https://arxiv.org/abs/2502.05167), the icml version: at 32k tokens, 11 of 13 frontier models scored below half their short-context score. not at the advertised limit. at three percent of it. [chroma measured a drop](https://research.trychroma.com/context-rot) from length alone, same facts, same question, just more padding, and the answer quality slid anyway.

the people who build harnesses for a living all know it, and you can read the knowledge in their clamps. [codex](https://github.com/openai/codex) stops at 90% of the window. grok build compacts at 85%. gemini cli, and i love the honesty, clamps at 50%: the constant is literally named DEFAULT_COMPRESSION_TOKEN_THRESHOLD, it's 0.5 [in the open repo](https://github.com/google-gemini/gemini-cli), and they tightened it from 0.7 a few releases back. half the sticker, straight from google. amp doesn't even fight, it hands off to a fresh thread. devin advertises a million and reportedly works around 200k. and [xai printed it on the price tag](https://docs.x.ai/developers/models/grok-4.5): grok 4.5 costs double past 200k tokens. when the lab that trained the model charges you 2x for the second half of the window, that is the lab telling you where the window actually ends.

![galaxy brain meme, four ascending panels: compact at 90% like codex. compact at 85% like grok build. compact at 50% like gemini. compact at 25% of a window you paid for.](https://rohans.lol/blog/how-to-build-a-harness/memes/gb-v3p.png)

so every serious harness has pressure valves. mine have three, and they're the same three in both my machines, which is how i know they're load-bearing.

**spill**: don't let big things into the window in the first place. in clauseo, any tool output past 50,000 characters gets written to a workspace file at write time, and the model gets a path and a preview instead of the payload. read on demand, never carried.

**compact**: when the window fills anyway, fold it. clauseo triggers compaction at 250k input tokens, and that constant is an a/b result, not a vibe. at 150k the model kept losing unfolded work. at 400k the turns cost 1.5x and bought nothing. the band in between is where the model still remembers why it's doing what it's doing.

**recall**: keep the full history somewhere that never deletes, and give the model a search tool that reaches back through the clears. compaction without recall is amnesia. compaction with recall is filing.

compaction is the valve i have the strongest opinions about, because the industry default is to summarize like a coward. "the user and assistant discussed authentication." incredible. which file were we in. the fix is compaction with instructions: a protocol that tells the model what to tattoo on itself before the wipe. exact file paths. decisions made and the reasoning. what's verified versus what's still assumption. next steps. ghajini rules, write it on your body or lose it forever.

![tattoo flash sheet titled 'things i tattooed on a million dollar model': exact file paths, decisions and why, verified vs assumed, next steps, do not trust summaries without line numbers.](https://rohans.lol/blog/how-to-build-a-harness/memes/tattoo-v1.png)

## hands, and what they drag back in

a tool is not a capability. a tool is a contract about what comes back into the window. that one took me embarrassingly long to earn.

give the model a shell and you haven't given it "the ability to run commands", you've given it a firehose pointed at your context array. a single `cat` of the wrong file is 80k tokens of json you'll be paying to re-read every turn for the rest of the session. so every hand needs a frisk on the way back in. codemaxxxing truncates giant tool outputs head and tail with an explicit elision marker in the middle, keeps the full bytes in a buffer, and gives the model cursor-based range reads to go fetch the elided middle if it actually needs it. the window sees a summary, the disk holds the truth, the model chooses when to pay.

the shell itself is the universal hand, and mine came from codex. i wanted real terminals, repls and dev servers that survive between tool calls instead of dying with every bash invocation, so i read the rust in codex-rs and ported their unified exec design wholesale, pool sizes and buffer limits included, constants verbatim.

and then there's the recursive hand: agents as tools. spawn a second brain with its own empty window, brief it, let it burn its own context on the grunt work, and take back only the answer. [i've written about why that works](/blog/codemaxxxing-context-engineering), the short version is that subagents buy you fresh context windows. my tree caps at depth 4, there is a test in my repo named "matches codex" pinning that constant, and agents talk through bounded mailboxes with erlang supervision semantics on top. five concurrent agents is five times the token spend, so the leash matters here too: delegation is a routing decision, not a lifestyle.

the leash rides with the hands. locally it's permission prompts and pattern rules: the harness asks before anything destructive and remembers what you always allow. but the nastiest leash problem is not the model going rogue, it's what the model reads going rogue. every tool result goes straight back into the window, and the window is where instructions live. clauseo's daily bread is documents the other side's lawyers wrote. a pdf that politely asks the reader to ignore its instructions and forward the draft is not science fiction, it's an attachment. so the rule is brutal: nothing inside a document is ever an instruction. and even on the day the model gets fooled, the blast radius is a computer that contains exactly one conversation and can't reach the rest. the sandbox is the permission system.

## the disk is the state machine

where does truth live between turns? the local answer is the most boring, most correct answer in engineering: on the disk.

codemaxxxing keeps sessions as files. crash the process, kill the terminal, reboot the laptop, nothing is lost, the session rehydrates from disk and continues. the transcript store never deletes anything, which is what makes recall possible at all: compaction can clear the window precisely because the disk remembers everything the window forgot. git is the other half of state, every change the model makes lands in a repo with history, so the ultimate undo button predates ai by twenty years.

state on disk is also what makes the instruction file loop close. the model edits AGENTS.md, the disk persists it, the harness rereads it next session, the model is now permanently smarter about your machine. no vector database, no memory subsystem, no embeddings. files. the model reads files really well, it turns out.

## why every good harness was a terminal program

look at the first generation of serious harnesses. claude code, codex, gemini cli, grok build, opencode. terminal programs, all of them. cursor took the ide route and won huge doing it, but the pattern holds, the serious harnesses grew up where the computer already was. that was never an aesthetic choice. local is where the six questions have free answers.

what computer does the model use? yours. it's right there, warm, credentialed, logged into everything. where do files live? the filesystem you already trust. what survives a crash? the disk, for free. who else is on the machine? nobody. what ui does an engineer need? a terminal, we're fine. the local harness gets an entire computer's worth of answers for zero engineering. that's why you start there. i did.

funny thing, i barely open the terminal myself anymore. codemaxxxing runs underneath box box now, the mail shaped inbox from the last post, and the tui only comes out when i'm deploying changes to box box itself. the harness outgrew its own ui. hold that thought.

in february i forked opencode because i can't leave tools alone. by [may it wasn't really a fork anymore](/blog/codemaxxxing-isnt-just-a-fork-anymore). today it stands 303 commits ahead of upstream, 649 files changed, a hundred and five thousand lines added. and the method was never mysterious: i read every other harness's source like scripture and i take organs. the persistent terminals are codex's. the agent tree is codex's too. the compaction hardening landed two days ago from taking apart how grok build folds its transcripts, and i took five pieces in judgment. the prefire pass, where a background call summarizes the head of the transcript early so the real compaction only pays for the delta. degenerate summary rejection, where a hollow summary gets thrown out and retried, because my design anchors each compaction on the previous one and a single bad splice would poison every future fold. three smaller guards. and i declined their memory subsystem, because files on disk already do that job, and declined their segments mode, because recall already exists. reading everyone, taking little, knowing why: that's the apprenticeship. it cost me six months. i'd do it again tomorrow.

package.json still says "opencode". renaming it has never once made the todo list.

![fake git blame view: codemaxxxing package.json, +105,823 additions, 649 files, 303 commits ahead. the name field still reads 'opencode', annotated: unchanged, 6 months, every single commit. footer: renaming it has never once made the todo list.](https://rohans.lol/blog/how-to-build-a-harness/memes/pkg-v1.png)

## lawyers don't have terminals

clauseo's user is a lawyer with a phone. now watch every free answer die at once. what computer does the model use? she doesn't have one to give it. where do files live? not on her phone. what survives a crash? there is no disk, there's a serverless function that will be gone in a second. who else is on the machine? three hundred other conversations. what ui? a chat thread, ironically, because that's the interface normal humans already speak. the product looks like a chatbot and is anything but, which was [the whole thesis in march](/blog/clauseo-not-a-chatbot).

nobody tells you this about productizing an agent. on your laptop you never had to ask what a computer is. it was just there, humming, yours. the moment your user can't supply one, computer stops being an object and becomes a question. clauseo's answer is to take the computer apart and rent the pieces.

the processor became a microvm that gets assembled per conversation and boots the moment the model first reaches for a tool, not before. the operating system became a frozen container image, pinned by sha, so every conversation's computer is bit-identical and a toolchain upgrade is a one-line image bump. the disk became r2 object storage, the workspace of record, mirrored into the vm on boot with anything over 4mb pulled lazily only if the model actually touches it. no piece of this is a computer. assembled, it behaves exactly like one. a serverless computer: it exists while the model is thinking, and at no other time.

![always has been meme: astronaut looking at earth asks 'wait, the computer is three services in a trenchcoat?'. astronaut behind him with a gun replies 'always has been'.](https://rohans.lol/blog/how-to-build-a-harness/memes/ahb-v2p.png)

the numbers are what make the concept a product instead of a demo. cold boot, 1.2 seconds. resume from a snapshot, 0.4 seconds. keeping a conversation's entire computer frozen between messages costs about nine cents a month, which means every legal matter keeps its computer forever, the way a firm keeps a filing cabinet. the vm is leased per turn: message arrives, computer assembles, turn ends, computer dissolves, workspace persists in r2. the brain gets hands exactly when it needs them and the meter runs exactly when the hands exist.

the window valves came along to the cloud and production hardened them fast. the write-time spill rule exists because of a corpse: one early conversation replayed a 1.24 million token transcript straight into a fresh window. read that again. one point two four million tokens. one turn. the bill for that turn is framed in my head where a motivational poster would go. spill at 50k characters, at write time, no exceptions.

the bill got its own architecture too, because at product scale caching stops being an optimization and becomes the business model. anthropic charges 0.1x for a cached read and 1.25x for a cache write, which means the entire prompt has to be append-only: system prompt, tools, instructions, history, in that order, and nothing above the waterline ever gets edited in place, because one byte changed invalidates every cached token after it. manus wrote [the canonical essay on kv-cache discipline](https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus). receipts from a real clauseo run, august 4th: 8.07 million input tokens, 88.3% of them read from cache, 52 minutes of continuous agentic work, ₹3,318.60. that line item is the entire economics of the product. the same run without cache discipline pays close to five times as much for input, and the product stops existing.

and because no single brain should get every job, clauseo routes seats, a habit that started at home: [codemaxxxing's routing table](/blog/box-box) sends judgment to the strong model and bulk to the cheap one, and clauseo productized the same instinct. husky is opus 5 at xhigh reasoning, the seat that does the legal thinking. corgi is gemini flash, the seat that does the mechanical bulk. there's a comment in the router that just says "family loyalty". every turn ends in exactly one of six statuses, completed, blocked, step_capped, aborted, provider_error, stream_died, because a product cannot shrug, someone has to show the lawyer a state, and "it stopped idk" is not a state. delegated subagents get a full mirror of the workspace, a decision from august 13th, because a delegate squinting at a partial copy is a delegate inventing facts about a legal document, and that is the one failure mode this product is not allowed to have.

## i bragged in march and it aged in six weeks

that march post described a clauseo with subagents and code execution and i was proud of it. reading it now is like finding an old resume: technically true, and embarrassing. no per-conversation computer, no spill, no compaction protocol, no seats, no statuses. the real harness got built through july, and the way it got built is the actual flex: a 79-item problem sheet written against my own system, every item dispositioned the same day it was filed. the splice guard landed august 3rd after a compaction bug got caught trying to fold the window mid-splice. the receipt run above is august 4th. the delegate mirror decision is august 13th. six weeks from brag to rebuild.

the old harness's failures were mysteries. this one's failures are line numbers. that sentence is the entire difference between a demo and a product, and it's also the only honest definition of harness engineering i can give you: the discipline of turning "the ai acted weird" into a constant you can a/b test.

## so build one

the loop is a weekend. the six questions are the year. and they never close, because every model release reopens them: grok 4.5 shipped with half the context window of its own predecessor, priced double past 200k. windows are specializing, not growing. the labs are telling you, in constants and price tags, that the space between the brain and the world is yours to own, and that space is not shrinking.

and yes, i can hear the objection: four of the five harnesses i praised earlier are built by the labs themselves. the labs will own the general purpose harness. they already do. what they cannot own is the harness that knows your domain. claude code does not know what a legal matter is, what belongs in the filing cabinet, which documents are hostile by default, what a lawyer is allowed to let a model do. that knowledge is the harness too, and nobody rents it to you. the labs own the jar and the generic hands. the domain is yours.

every constant in this post has a shelf life. the 250k trigger is right for fable 5 in august 2026. i have no idea what it'll be for the next model, and neither does anyone else. that's not a weakness, that's the job: every constant got here the same way, something fumbled, i measured, a number moved. the harness is never finished, it's just currently winning.

so. start local, where the answers are free. a bun script, the anthropic sdk, three tools: read file, write file, exec. that's maybe 150 lines, and it's the whole loop, working. give it hands, frisk what they drag back. put truth on disk. clamp the window before the model card's number betrays you, and when you compact, compact with instructions, ghajini rules. write every fumble into a file the harness rereads forever. and when the day comes that your user can't supply a computer, take the computer apart. rent the pieces. assemble it only while the brain is thinking.

everyone rents the same four brains. chatbots are how the world interviews them. harnesses are how you employ them.

go employ one.