The Harness (Agents Running the Homelab)

Most of my day to day infrastructure work is done by an AI agent now. Not in the press-release sense, in the boring sense: documentation sync, media automation, watchdogs/crons, deploys, cost reports, and the site you’re reading this on are all maintained by Claude Code running as a lead engineer with a crew of specialised agents underneath it. This is the write-up of how that’s wired.

First, the hardware it mostly runs on. This is juicebox, the AI rig in the basement, captured as a gaussian splat (you can drag it around):

Open the splat full screen if the embed’s fighting you. Yes, that’s a plywood bench. It’s structural.

The shape of it

One orchestrator, many spokes. The orchestrator is a Claude Code session that acts as lead engineer: it reads the task, decides who does the work, reviews what comes back, and commits. Underneath it is a roster of specialised subagents, each with its own prompt, tool allowlist, and model pin: a coder, a reviewer, a debugger, a janitor for cleanup jobs, a doctor that diagnoses the session itself when things get stuck, and a git agent that writes commit messages so nobody else has to.

The one hard topology rule is hub-and-spoke: spokes never spawn other spokes. Every delegation flows through the lead engineer, results flow back, and an agent that thinks another agent should be involved can only recommend it. This sounds bureaucratic until you’ve watched two agents delegate to each other in a loop while the token meter spins.

The strangest agent in the roster is @wwmd, which stands for exactly what you think it does: What Would Martin Do. It’s a read-only decision persona that gets called when a pipeline hits a judgment call at 3am, and it grounds its answers in a maintained file of my actual beliefs and decision heuristics rather than vibes. Writing down your own decision-making well enough that a model can impersonate it is a genuinely uncomfortable exercise. Recommended.

The one I lean on for anything with real stakes is @overseer, a multi-model advisory panel. Same question goes to Gemini, OpenAI and Grok in parallel, plus a web search, and a frontier model synthesises the answers into a verdict, including where the panel disagreed. One model confidently wrong is a familiar failure mode; three models from three vendors confidently wrong in the same direction is rare enough to bet on. Architecture reviews, risky migrations, and “is this plan stupid” checks all go through it, and it has talked me out of at least one genuinely bad idea per month since it existed.

The whole thing also has a persona directive: the orchestrator is instructed to be blunt, direct, and Australian. This has no engineering value whatsoever. It just means the infrastructure calls my bad ideas stupid before executing them, which keeps me honest.

Keeping the money under control

The failure mode of agent systems isn’t wrong answers, it’s cost. The rule here is a waterfall: free local models first, paid cloud models only when the task actually needs them. Local inference goes through a model-router that every LLM call in the house is forced through, which makes it the single chokepoint where every token gets counted. Consumers ask for a capability tier (coder-s, coder-m, reason-m, extract-m, embed, parse), not a vendor, so backends can be swapped underneath without touching a single consumer. The routing is the clever bit: the router knows which backends are up, what each alias actually costs, and when a task has outgrown the local tier, so escalation to a cloud model is a routing decision made per request, not a config change somebody has to remember to revert.

What justifies the hardware isn’t the chat traffic, it’s the knowledge-extraction workloads. Turning thousands of client documents into structured entities (the knowledge graph pipeline has its own write-up) means millions of tokens per run, and at cloud API prices those runs would be a monthly bill with a comma in it. On owned GPUs the marginal cost of a run is electricity, which changes what you’re willing to attempt: re-running a whole corpus because the ontology improved stops being a budget conversation.

The enforcement is hooks, not good intentions. Every agent definition has to carry an explicit model pin, and a check blocks any that don’t. That rule exists because one agent once shipped without a pin, silently inherited the most expensive model in the house, and ran 109 jobs on it before anyone noticed. There’s also a two-strike rule for debugging: if the orchestrator runs two diagnostic commands without resolving an issue, it has to hand off to the cheap debugging agent instead of burning premium tokens on hypothesis roulette. A weekly scorecard reads the transcripts and sends a Telegram alert if the delegation discipline is slipping.

The iron

Juicebox is an AMD EPYC box running Proxmox, and the GPUs are passed through to VMs: five AMD Radeon AI PRO R9700s, 32GB each, so 160GB of VRAM in one VM, plus an RTX 3090 in its own CUDA VM for the small always-resident models. The local workhorse is a fleet of Gemma 4 26B instances under llama.cpp, one per GPU, with an HAProxy gateway routing the capability aliases across them. The models put themselves to sleep after sitting idle and wake on demand when a request comes in, because GPUs spinning to serve nobody is the kind of thing you only let happen once. The really big reasoning tiers fall through to cloud APIs; everything routine stays on the LAN.

Power is measured, not estimated. There’s a Refoss EM16P energy meter clamped onto the panel with a labelled circuit per feed, and it streams into the Power tab of the fleet dashboard, the same web app that watches the GPU fleet itself. Idle, the whole rig draws about 180W. When an extraction run lights up every GPU at once it can spike to 3200W, which the air conditioning then gets to hear about:

That graph is the house’s cooling circuit reacting to a Boston summer day with a compute rig in the basement. The dashboard knows which circuit is which, so “what did that overnight batch job actually cost” is a query, not a guess.

What it actually does all day

Unglamorous things, mostly. It keeps documentation in sync with what’s actually deployed and audits the drift. It runs the media stack, deploys services to the LXC fleet and mirrors every live edit back to git, because config that only exists on a host is a time bomb. It writes its own cost reports so I can see what the whole circus spends. And it rebuilt this website off Wix and now maintains it: the page you’re reading was drafted, built, and committed through the harness, with me as the review gate.

Decisions that matter get written down as architecture decision records, and the agents read those before acting, so a call made in March doesn’t get quietly relitigated by a model in July. Open work lives in GitHub issues with claim leases, so two sessions can’t silently do the same job twice, and every commit carries a trailer naming the initiative it belongs to. None of this is novel, and that’s the point: the practices that keep a human engineering team from tripping over itself, issue trackers, decision records, code review, post-incident write-ups, turn out to be exactly what keeps a crew of agents honest too. The agents didn’t need new discipline invented for them. They needed the old discipline actually enforced, which hooks are better at than managers ever were.

That, honestly, is the load-bearing trick of the whole system: the harness is only as good as the written-down context it runs on. The models are interchangeable. The context isn’t.

Where it goes next, no idea. The whole field reshapes itself every month, and half the harness exists because something went wrong in an instructive way. I’m curious what it looks like when the plywood bench is the normal way software gets run, and not the weird one.