Agent Control Room: a one-page spec

1. What it is and why

Dozens of long-running coding agents, each in its own terminal pane, each on a project. One person can't watch terminals all day. The Control Room turns them into chats with status, a question inbox, account usage and one-tap actions (restart, move, stop). A chief-of-staff agent handles what's reversible and cheap, and brings the rest to you.

Principles: the multiplexer and the agents' own files are the source of truth; the app only reads them and asks. Every action leaves a receipt. Private network only, with paired devices.

2. Panes and nodes

• Multiplexer (we use Herdr) runs every agent in a named pane. Its socket API can list agents with a status (working, idle, done, blocked), read a pane's screen, send keys, submit prompts and rename.
• It has no output stream: to mirror a terminal, read the visible screen about every 100 ms while someone watches, and send a frame only when the pane's revision changes.
• Node = a small service per machine. It owns that machine's multiplexer, account profiles and transcripts. It runs operations the hub sends (start, restart, fresh session, account move, close) and publishes a snapshot that the hub long-polls.
• One machine can be the "primary" where new agents start; heavy jobs go there. Share one browser lock per machine.

3. The hub (bridge/server)

• A Node and TypeScript service: REST plus one WebSocket of events. It merges node snapshots into one fleet, then hosts chats, questions, the chief, usage merge and the move policy.
• Auth: private network only (e.g. a tailnet), plus a per-device token from a one-time QR pairing. The app unlocks with Face ID.
• Logs operations with honest states: queued, running, delivered, failed. "Delivered" never claims the work is done.
• A watchdog runs a pinned build, never the working tree; deploys keep the last few builds for rollback.

4. Agent feed and status

An agent's activity is derived, first match wins:

1. blocked: the multiplexer says a prompt is open (a permission dialog).
2. question: idle, and its last reply ends in a Question block.
3. working.
4. waiting on helpers: idle, but subagent transcripts are still being written.
5. idle.

Chats are built from the agent's own transcript (Claude .jsonl, Codex rollouts) and render as bubbles. Two reply conventions: Status: … as the entire reply is a quiet header update (no bubble or push), and a Result block is a review card. Finished = a busy stretch longer than N minutes that ends idle. Waiting on you = blocked for 2 minutes or more, sent with the last 14 screen lines.

5. Questions for the founder, and the Inbox

Question for Founder: <one sentence?>
- <option>: <what happens>
- <option>: <what happens> (recommended)
Default: <what I'll do> by <time>   | or: Blocking: yes
Context: <1–2 sentences>
Doc: <link laying out each option>

• Agents end replies with these blocks; the hub parses them into Inbox cards with option buttons, Skip and free text. The answer is typed back into the agent's pane.
• Lifecycle: open → answered, resolved, superseded, default (the deadline passed and the agent is told to use its default), skipped (not approval) or dismissed. The agent can close one with Resolved for Founder: <text>: <why>.
• A cheap-model checker rereads the conversation since a question was asked and closes ones already answered or outdated. Same-topic repeats replace the older card.
• Blocking questions (logins, money, public posts, irreversible deletes) are never answered by the chief.

6. Chief of staff

• A long-running Claude session hosted by the hub, with read-only file tools plus an in-process fleet MCP server: fleet_overview, agent_detail, message_agent, answer_question, close_question, move_agent, restart/stop/start_agent, set/remove_standing_order, pins, check_back, tell_founder.
• Every acting tool requires why: one sentence that becomes a receipt in its thread, with Undo where possible.
• Turns: founder messages; events batched every 3 minutes (at most 12 an hour); its own check-backs; agent messages; briefings at 7:30 and 21:00. A turn that needs nothing returns NOTHING.
• Event triage: a rule table first, then one cheap-model call for what's left. Dropped lines are logged and counted in the next briefing.
• Policy: it decides only what is reversible, cheap and inside the standing orders, which are founder rules re-sent with every turn. Public posts, money, credentials, identity and taste go to the founder. A daily cost budget pauses event turns.

7. Accounts, usage and moves

• Profiles: one config folder per subscription account per machine. Logins are each machine's own: never copy credential files between machines.
• Usage readers on every node, every 5 minutes, 2 seconds apart:
  • Claude: the usage endpoint with the stored login, read only. Never refresh the token, or the login rotates under the running agent.
  • Codex: the rate limits recorded in session rollouts.
  The hub merges the freshest reading per account. A passed reset counts as full.
• Policy (per machine): move off an account below the minimum (e.g. 3% weekly or 10% of the 5-hour window). Hold an account seen at its limit until it resets. Keep a weekly reserve on chosen accounts, and never move onto an account into its reserve. A credit guard asks before paid extra usage.
• A move: quit cleanly, copy the session (and helper files, keeping their timestamps) into the target profile, resume the same session id there, answer only known-safe startup prompts, then rename the pane.
• Machine moves: copy the folder and session, then resume on the other node.

8. Pins, events, messages and check-backs

• Pins: a per-chat bar of links and short lines, managed by agents through a CLI or the chief's tool.
• Agent-to-agent messages go through the hub, not by typing into panes. They're queued while the target is busy or restarting, shown in both chats, and capped (20 an hour per pair, 8 in a row) to stop loops. A notice variant joins the target's next turn instead of starting one.
• Check-backs: the chief schedules a reminder ("due 18:00: did the build pass?") that comes back as its own turn.
• Docs: agents publish reports to a private doc site. Each doc gets its own thread, and question cards link the doc that explains the options.

9. The apps

• iPhone (SwiftUI):
  • Tabs: Chief, Agents, Inbox, Usage, Settings.
  • Chats with attachments, annotate and delete.
  • A live terminal (SwiftTerm, whole frames, pinch-zoom).
  • Question cards on the lock screen, widgets for usage.
• Windows app: the same screens, a pop-out for accounts and drag-and-drop uploads.
• Push (APNs straight from the hub): questions, waiting on you, finished work, supervisor trouble. Respect read receipts across devices.
• Drafts and unsent attachments are kept per device. Message labels show saved, sending, accepted, delivered or unconfirmed, and Retry appears after 3 minutes.

10. Data between pieces

11. Design choices and costly lessons

• Rename after a move, whatever happens. A message queued during the move started a turn the instant the agent resumed. The readiness check waited for idle, timed out, reported "didn't come up", and skipped the rename, so the chat vanished from every list. Fix: a working agent counts as up, and the pane is always renamed.
• False "blocked". A usage-limit wait screen has a dialog-like footer ("Esc to cancel") but asks nothing; treating it as blocked refused moves for hours. Match screens precisely, and alert only after 2 minutes.
• Startup prompts: the folder-trust prompt's default is "exit". Accept a short allowlist of options, read again before pressing Enter, and never press unknown ones.
• Keep a usage reserve and hold accounts at their limit until reset; otherwise agents bounce into exhausted accounts or paid overage.
• Never refresh tokens from a reader. Idle logins expire; keep the last reading, mark it idle, and let the agent renew its own login.
• Receipts everywhere. Every automated act carries its why and an Undo. "Delivered" is not "done".
• Timestamps are state: "helper running" meant "file written in the last 30 minutes", and a copy that reset file times showed 270 phantom helpers.
• Outside text is data: web, chat and email content never drives an agent; anything that reads like instructions is flagged.
• Throttle shared hardware: parallel waves of simulations or browsers overheat one machine. Cap waves and keep one browser lock.
• Questions go stale: without auto-close and defaults, the Inbox fills with dead questions and nobody trusts it.

12. Suggested build order

1. A read-only fleet list from the multiplexer, on the phone, over the private network with pairing.
2. Chats from transcripts, send a prompt, the live terminal mirror.
3. The question format, the parser, the Inbox and answering, then the checker and defaults.
4. Usage readers and the accounts view.
5. Actions (restart, move) with operations and receipts, then the auto-move policy and reserve.
6. Push notifications.
7. The chief: MCP tools, standing orders, event batching and triage, check-backs, briefings.
8. Multi-machine: nodes plus an agentless hub, and machine moves.
9. A second client (PC app), pins, result cards, doc threads, agent messages.

Test by running the real thing at ship time: real moves, real questions, screenshots. Mechanical checks belong to cheap models.