Buzz Coding Agents, Part 2: Adding Agents on a Homelab

Part 2: adding Buzz coding agents on a homelab

Introduction

In part 1 I wrote about self-hosting Buzz, Block's open source chat workspace for humans and coding agents, on a small Hetzner VPS.

That post ended on an open problem. Each agent container had about 1 GB of memory and one core, and when a build went over that, the kernel's OOM killer took it out silently. In the chat the agent still looked like it was thinking.

I didn't move off the VPS. I added three more agents on a homelab machine and kept the VPS ones, so the pool is bigger and split by job. The homelab agents do the heavy work: long-running tasks, real builds, emulator runs, browser tests. The VPS agents stay for quick asks and orchestration.

The relay itself never moved. It's a chat server, and it was never the bottleneck.

Why more memory on the VPS wouldn't have fixed it

Some of what I wanted agents to do wasn't a tuning problem.

• Android emulation needs KVM. Booting an Android Virtual Device for real flutter test runs, not just flutter analyze, needs hardware virtualization. Most budget VPS plans don't expose nested virtualization, and mine didn't.
• Headless Chromium gets flaky under memory pressure. Playwright screenshots and console-error checks would sometimes fail or time out once the container was near its limit. I could never tell if I was looking at a real bug or at resource starvation.
• Every agent carried its own toolchain. Each Docker container had its own Node, Flutter, and Android SDK. That's fine for one agent. It adds up fast at three, and a stale SDK image became one more thing to rule out when something didn't work.
• I want proof, not a summary. I want every task verified with a real build, plus captured screenshots and video, published to an internal site I can review later. That doesn't happen in a 1 GB container.

The homelab box

The homelab is a machine with 32 GB of memory and 8 cores running Ubuntu Server. No desktop, I administer it over SSH. It sits on my home network, and I reach it from outside over Tailscale instead of forwarding ports on the router.

Native processes instead of containers

The homelab runs Flutter, the Android SDK with a pre-built AVD, a JDK, and Node under ~/sdk, with no sudo and no per-agent image. Three agents run on top of that as long-lived processes, while the VPS agents keep running in their containers:

I went native instead of one container per agent for a boring reason. Three images would each carry their own copy of the Android SDK and Flutter, and I'd have to get KVM into every one of them. Running natively means one emulator, one Chromium install, one node_modules cache, and I only had to figure out the emulator once.

Each agent still gets its own identity: its own Nostr keypair, its own systemd --user service, and its own env file at mode 600.

systemctl --user status buzz-opus48 buzz-terra buzz-bob
loginctl enable-linger roman

enable-linger is the part I'd forget. Without it the user services stop when you log out, and the agents quietly disappear from the channel after a reboot.

How they work with each other

They talk to each other in Buzz, same as they talk to me. Mostly that means one agent reviews another one's work, which is the thing I'd otherwise have to do myself on every change.

The VPS agents didn't retire, they just changed jobs. They handle VPS-specific work and orchestration. A typical flow is me asking a VPS agent to go through open tickets, and it hands the real work to the heavy agents on the homelab.

What I still need to figure out

The setup runs, but four things are still open.

Distributing work. Right now I assign tasks by typing mentions into Buzz. That doesn't scale past a handful of tasks, and nothing tracks what's assigned to whom. Gastown looks like the closest thing to what I want: work items with dependencies, short-lived worker agents each in their own git worktree, a reviewer agent, and all the state kept in the repo so it survives a restart. I haven't decided whether to adopt it or write something smaller myself.

Screenshots don't look right on Linux. Rendering on the homelab isn't as nice as on the Mac. Fonts are the obvious difference, and the captures I get back aren't something I'd want to publish. I either need to pin a font stack on the homelab and accept it looks different, or send the screenshot and video jobs back to the Mac agent and leave builds and tests on the homelab.

Seeing what an agent is actually doing. This was part 1's complaint and it's still true. I can check that a service is alive, but I can't see the current state of an agent or the sessions it spun up. systemctl tells me the process exists, not what it's working on.

Whether Buzz is still the right place for this. For talking to agents from my phone, it's good, and I'm not replacing it. But I think I've been using a chat app as a work tracker, and that's where most of these problems come from. Chat and work state probably want to be two different things.

What's next

In the next part I'll write about how I manage work for the agent army, without typing everything into Buzz by hand.

---