/home/michael

Harness engineering

4 min read

Note

The harness is the architecture, not the oversight

Harness engineering is the practice of building the full architectural scaffolding within which AI agents operate — structured documentation they can reason with, constraints that enforce invariants, and feedback loops that let them know when they’ve succeeded. It is distinct from prompt engineering (which shapes individual tasks) and from oversight (which monitors outputs after the fact). The harness is the infrastructure that makes delegation coherent at scale.

Harness engineering

One-sentence definition: A harness is the full architecture — structured documentation, enforced constraints, and feedback mechanisms — within which AI agents operate reliably without requiring continuous human supervision of individual decisions.

The term comes from software engineering. Ryan Lopopolo, writing about building a product with coding agents at OpenAI, described the harness as the difference between handing an agent a task and building the conditions under which the agent can work productively over time. The observation: “building software still demands discipline, but the discipline shows up more in the scaffolding.”

The harness has four components:

Agent-legible documentation — structured material that agents can navigate, not just retrieve. What agents can’t see doesn’t exist. The analogy used is a table of contents rather than an encyclopaedia: progressive disclosure, not exhaustive coverage. Agents need to know where to look, not to have everything at once.

Architectural constraints — rules enforced at the system level, not negotiated per task. These are invariants: boundaries that hold regardless of how the agent interprets a given instruction. They reduce decision space, which improves reliability and makes outputs more evaluable.

Feedback loops — mechanisms by which agents learn whether output met the goal. Human review is one form, but not the only one. Computable success criteria — tests, validators, checks that run automatically — close the loop without requiring a person to evaluate every output.

Garbage collection — recurring cleanup processes that prevent accumulated semantic and structural debt. Without these, quality degrades silently as edge cases accumulate.

What it’s not

Harness engineering is not prompt engineering. Prompt engineering operates at the level of individual instructions; the harness is the environment in which those instructions are given. A well-crafted prompt inside a weak harness will still produce inconsistent results across parallel workstreams.

It is also not oversight in the conventional sense. Oversight is retrospective — catching problems after they occur. A harness is prospective — structuring conditions so that the range of possible outputs is bounded before work begins. Governance built only on oversight scales poorly with the number of agents running in parallel; governance built on a harness scales with the architecture.

What it requires in higher education

At the individual level, the three prerequisites for effective agent-first workflows — a plan before handoff, documentation treated as infrastructure, and domain expertise sufficient to evaluate outputs — constitute an informal harness. They work because one person holds the architectural constraints in their head.

At institutional scale, that breaks. Multiple people directing agents without shared constraints produce compounding drift that no individual reviewer can track. The institutional equivalent requires the harness to be externalised: shared documentation structured for agent consumption, quality criteria written as computable rules, and feedback mechanisms that close the loop across parallel workstreams rather than within a single working session.

This is what computable governance points toward: governance structured so it can be computed against, not just interpreted by committees. It is an architectural question, not a policy question.

Sources

Notes

  • See agentic workflows for the broader context on agent-first working patterns.
  • The three prerequisites in Working effectively with AI agents (plan, docs-as-infrastructure, domain expertise) are the informal version of a harness; this note extends that concept to the institutional level.
  • vibe-coding is the failure mode that a harness is designed to prevent — at the institutional level, the institution without a harness is vibe coding at scale.

Continue reading

  • AI agents

    An AI agent is a system that autonomously executes multi-step tasks using language model reasoning — distinct from an AI assistant, which responds to individual prompts. Agents plan, act, observe results, and adapt, using tools such as file access, code execution, and web search. They perform best when given clear goals, explicit constraints, and well-prepared context.

  • Agentic workflows

    A mode of working in which AI agents handle execution-level tasks while the human operates at the direction layer — deciding what should be made, defining what good looks like, and evaluating outputs.

  • Vibe coding

    Vibe coding describes using AI tools without maintaining genuine accountability for what they produce; accepting outputs without the scrutiny, direction, or judgement needed to evaluate and improve them. Simon Willison drew the key distinction: vibe coding versus vibe engineering, where the latter uses the same tools while remaining genuinely accountable for every output.

Graph View