Agent Harnesses — The Model Was Never the Bottleneck

I watched a talk by Tejas Kumar where he pulled off something that looks like a magic trick and is actually the whole point of agent engineering. He took one of the worst models you can still rent — gpt-3.5-turbo — gave it one fixed prompt and one job, and made it reliably do that job. The trick: he never improved the prompt, and never swapped the model. He improved everything around it.

That "everything around it" finally has a name. An AI harness is, in Tejas's framing, everything except the model weights — the tools the model can call, the context you feed it, the guardrails that bound it, the verify step that checks its work, the loop that ties it together, and the environment it runs in. The model is a rented brain. The harness is the body, the senses, and the supervisor. Let's build one, piece by piece.

What a harness actually is

When an agent fails, the reflex is to reach for a bigger model or a cleverer prompt. The harness mindset says the opposite. Mitchell Hashimoto — who coined "harness engineering" in early 2026 — put it as a rule: whenever an agent makes a mistake, you engineer the environment so it can't make that mistake again. The fix is rarely "try harder"; it's "what capability is missing." Those capabilities are the parts below — tap any one to see what it does in this repo.

Two different things are both called a "harness"

One quick piece of vocabulary, because it trips everyone up. The word points at two unrelated things — keep them apart.

Same word, opposite jobs. One grades the model; the other employs it. This whole post is about the second kind.

The experiment: one model, one prompt

Here's the setup that makes the idea undeniable. The repo is a sequence of git branches. The model is pinned to gpt-3.5-turbo-0613 the entire way — chosen because it's weak. The task never changes: go to Hacker News and upvote the highest-ranked story you haven't voted on. And the system prompt and task text are identical from the first branch to the last.

So every improvement below comes from the harness — never the prompt, never the model. The agent travels from "loops forever and lies about success" to "verified, recovers from a login wall, stops the moment it's done."

Build the harness, step by step

This is the heart of it. Click through the five branches — each one adds a small, boring, deterministic capability, and the diff is tiny every time. Watch the agent get reliable without the prompt ever changing.

while (true) {
  const res = await model.chat(messages, tools)
  // the only exit: the model itself says it's done
  if (res.finish_reason === "stop") return res
  messages.push(...await runToolCalls(res.tool_calls))
}

// agent/4-guardrails.ts
export const defaultGuardrails = combineGuardrails(
  maxIterations(15),   // stop runaway loops
  maxMessages(50),     // stop runaway context
)
// agent/7-index.ts — the ONLY change vs branch 0:
await runLoop(MODEL, messages, tools, defaultGuardrails)

// agent/7-index.ts — 20 lines collapse to 2
const result = await runHarness(TASK, MODEL)
printHarnessResult(result)
// runHarness opens the browser, builds tools + context,
// runs the loop, and always closes the browser — even on error.

const result = await runHarness(TASK, MODEL, {
  verify: verifySuccessfulUpvote,   // reads the trace, not the prose
  maxAttempts: 3,                   // retry until it really happened
})
// VerifyResult = { passed: boolean, reason: string, fatal?: boolean }

// agent/6-harness.ts — success short-circuits the loop
const guardrails = combineGuardrails(
  stopAfterUpvote(() => upvotedStory),  // → stoppedBy: "success"
  defaultGuardrails,
)
// login-handler.ts fills the form, then injects:
// "You are now logged in. Finish the task."

Guardrails catch crashes. Verify catches lies.

The most useful thing I took away: guardrails and verify are not the same tool, and you need both. They fail-stop on completely different things.

Without a verify step, a model that hallucinates "Done!" looks identical to one that did the work.

Because verify reads the trace instead of the answer, the whole run is auditable. Here's roughly what a successful attempt prints — the verdict is derived from the tool calls, not from anything the model said:

Your turn — spot the missing capability

One quick check to make sure it stuck. An agent finishes a run and reports: "Done! I upvoted the top story." But when you look, nothing was clicked — it hallucinated the whole thing. Which harness capability catches this?

What I took away

I came into this thinking model choice was most of the battle. The repo is a tidy proof that it usually isn't. A genuinely weak model, held fixed, became reliable through five small deterministic additions — none of them prompt-whispering. When my own agents misbehave now, my first question has changed from "is there a better model?" to "what capability is my harness missing?" — a context limit, a guardrail, a verify step, a recovery path. That reframe is the whole lesson, and it's a far more tractable problem than hoping the next model drop fixes everything.

The model was never the bottleneck. The harness was.

Resources & References

Start with the 20-minute talk this whole post is built on, then go deeper with the canonical guides from the teams actually shipping agents at scale.