ADR-002 — Dedicated home — iron & service topology

Status: Proposed (design-panel synthesis) · Date: 2026-07-22 · Track: E1 (dedicated iron — Pillar 5) Method: adversarial design panel — estate-scout + four thinker-canon lenses (cloud-architecture/load-shape · observability/saturation · simplicity · data-durability). Dissent preserved below.

Follows the ADR-001 template. Brand-free: iron, stores, and services named by capability only.

---

1. Context

The foundation spec (Pillar 5) commits the platform to its own dedicated iron — off the general estate box — so its three spiky internal workloads (live-event video, AI inference, realtime) never compete with the rest of the estate. §9 leaves the shape open: "one bigger box vs. a small fleet (learning / community+realtime / AI split)." E1 decides that shape.

The load is known and modest: tens → low-thousands of members, a small ops team with no night shift. The stateful heart is already locked by ADR-001 — an append-only conversation log as source of truth (I9), a relational store, a cache/session store, an object/media store (delivery decoupled — a paid host is a URL repoint), an AI index, and the identity store. What is not yet decided is where those bytes physically live, what shares their failure domain, and where the two ruinous-to-own-idle workloads (GPU inference, live-event video egress) run.

The estate-scout confirmed E1 is mostly greenfield iron — the server, service composition, store provisioning, and backup wiring do not exist yet. What the estate does supply is the operate/deploy/roll-back doctrine above the iron: a hook-injectable deploy wrapper with fail-closed smoke + auto-revert, host-hardening/secrets/ingress/backup/observability runbooks, and a push-to-deploy pipeline already staged for exactly this host (down to a live healthcheck fix). That layer is REUSE, not rebuild.

2. Decision drivers

• Isolate the spike (live-event egress, AI inference) from the stateful core — the whole reason E1 exists.
• One consistency + backup domain for durable state — the log is the moat; fragmenting it or under-protecting it is the one unrecoverable mistake.
• Small ops team: every always-on stateful node is a permanent 3am liability; ops-surface, not compute, is the binding constraint at this scale.
• Reversibility: grow into more iron by measured repoint, never by standing up a distributed system on a diagram's say-so.
• Own-idle avoidance: the workloads to isolate are exactly the ones that are ruinous to own sitting dark.

3. Options considered

• A — Co-tenant on the existing estate box (a new stack behind the estate's ingress; no new iron).
• B — One new dedicated box (own iron; all workloads — learning · community+realtime · AI/GPU · stores — co-resident, one ingress, one identity).
• C — Small fleet split by workload (separate nodes for learning · community+realtime · AI-inference-GPU).
• D — Hybrid: own the stateful core on one box, rent the spiky stateless edges (GPU inference + live-event video delivery/CDN burst off-box, released when idle).

4. The panel (rankings diverged; read the core argument, not the rank)

Lens	Rank #1	Core argument
Load-shape (cloud-architect)	D	The cruel mapping: the workloads you MUST isolate (video egress, inference) are exactly the ones you must NOT own (spiky, idle-expensive, elastic); the workload you SHOULD own (log + relational + cache + media origin + steady community app) is the flat, cheap one. Reconcilable only by owning the steady part, renting the spike — not a bigger box, not a fleet. Don't buy a GPU, don't buy egress.
Saturation (Gregg)	D	The live-event spike is a bandwidth + connection-concurrency problem, not a CPU one — a fleet that keeps egress on your own uplink saturates the same pipe. Only D moves the actual saturating resource off owned iron. But you're sizing iron for a load you haven't instrumented — wire U/S/E + RED and run a synthetic live-event load test before buying the box.
Simplicity (Torvalds)	B	Ops-service-count × team-size is the ballgame. One box off the estate satisfies the hard isolation constraint at minimum ops surface. A workload-fleet is premature generalization — 3× the patch/backup/triage surface for a load one box holds for a year. B is D at an earlier point in time: grow to D by peeling the spike off first, when a measurement forces it. Verify whether "AI inference" is even new GPU iron or just the estate's existing brain.
Data-durability (Kleppmann)	D	"Split by workload" (C) cuts across the grain — it fragments the source of truth into two authoritative relational stores keyed on the same member sub, inviting a replicated-mutable-tier leak (violates I9). Draw the fault line at stateful-vs-spiky: one owned consistency/backup domain for all durable state; rent only stateless-degradation workloads. And "one box" ≠ "one disk" — the log must not share a failure domain with its only durable copy.
Estate-scout	—	E1 iron is greenfield; the estate supplies the operate/deploy/roll-back doctrine (deploy-wrapper auto-revert + this host's staged pipeline = REUSE), an adaptable agent control surface (dry-run + host-allowlist + .bak), and a read-only console skeleton (wrong data plane) — none shorten standing up the iron, all shorten operating it.

5. The convergence (what all five agree on, under the different ranks)

Despite ranking A/B/C/D differently, the lenses converge on one architecture:

1. Kill co-tenancy (A). Every lens agrees with Pillar 5: co-residence gives the video spike and inference a path into the log's fsync/durability latency and shares fate with the estate. Dead on arrival.
2. Kill the workload-fleet (C) at this scale. Torvalds (ops tax) and Kleppmann (fragmented source of truth + tier-leak surface) arrive from opposite doors at the same verdict, and Gregg adds the killer: a fleet split by workload doesn't even solve the egress spike — video still exits your uplink. C is isolation theater: most ops surface, least isolation of the actual saturating resource.
3. The fault line is stateful-vs-spiky, not workload-vs-workload. Kleppmann names it; the load-shape lens confirms it (own the flat, rent the bursty); Torvalds confirms it (the only split that pays for itself is the genuinely different, genuinely spiky resource profile). Split by statefulness, never by "learning / community / AI."
4. Own the steady stateful core on one box; rent the spiky stateless edge off-box. Isolation is bought by elasticity (rent and release the spike), never by buying a bigger box (B "isolates" a spike that still lives inside it) and never by owning a GPU that idles 95% of the week.
5. B and D are the same architecture at two points in time. Start on one owned box carrying all durable state; promote the spike (video delivery, GPU) off-box the day a measured saturation threshold forces it. Growth is a repoint, not a rebuild — the media-delivery repoint the spec already anticipates, extended to inference.
6. Instrument before you size. You cannot pick or size any option honestly without U/S/E on NIC/proxy/stores and a synthetic live-event load test. The spec names an observability estate and uses none of it in E1.
7. Durability is a separate axis from topology. "One box" is only safe if the source of truth has an off-box, restore-tested survival path. An untested backup of the log is exactly as real as an unenforced invariant: a wish.

6. Decision — "Own the stateful core · rent the spike · grow by repoint"

A hybrid that takes the winning move from each lens: ship B's single owned box today, architected as D, so the promotion to rented edges is a repoint the day a measured threshold demands it.

6.1 Own (one stateful core — one consistency + backup domain)

• One dedicated box off the estate carries all durable state: the append-only conversation log (single authoritative writer per conversation; seq store-assigned, never wall-clock — ADR-001 I9), the relational store, the cache/session store, the object/media origin + metadata, the AI index, and the identity/session store — behind one reverse-proxy ingress and one identity provider, deployed over the private tailnet, borg-backed.
• Tier has exactly one authoritative home (the identity store), read at request time — never a durable, cacheable column in the learning core's or community's own store. This is the topology-level enforcement of I9; it is why the fleet (C), which forces a tier copy into a second store, is rejected.
• Media is addressed by content-hash / URL-indirection through a storage abstraction, so the origin can move (to a CDN, another disk, an object host) without rewriting a single reference. The compute is reversible; the addressing of the stateful bytes is the thing you pay for later if you get it wrong now.
• created_at is display-only; seq is authoritative for ordering — baked in now so a later edge expansion can never reintroduce a wall-clock ordering dependency.

6.2 Rent (the spiky stateless edge — hold no source of truth)

• Live-event video delivery points at a CDN + edge/object store the moment concurrent-player demand crosses the box's measured bandwidth/connection ceiling. Media bytes live in the origin we own; delivery is the decoupled repoint. The saturating resource (egress + connection table) never terminates on the stateful box during an event.
• AI inference runs off-box on burstable capacity — on-demand GPU-hour or a no-retention inference endpoint — released when idle. The AI index stays on the owned box; only the prompt + retrieved context transits to inference. We do not buy a GPU and we do not buy egress at this scale. Their worst failure is degradation ("AI slow / video buffering"), never corruption ("log lost").

6.3 Defer (Torvalds' cut — scoped, not killed)

• No fleet. One box until a named saturation metric crosses a written threshold — not a topology diagram, not a calendar.
• Defer the warm standby / replica until membership justifies it — but do not defer the off-box, restore-tested backup of the source of truth (§8 P0). The replica is availability; the drilled off-box copy is survival, and survival is non-negotiable from the first member write.
• Investigate before provisioning any GPU-hour (acting CTO, per Torvalds): Pillar 4 says the AI half runs on the estate's existing governed brain + agent-governance plane. If inference is entirely an estate seam, the only spiky thing E1 owns is live-event video, and even D shrinks to "one box + rented transcode/CDN at event time." Measure the real inference footprint first.

7. Tradeoff matrix

Dimension	A co-tenant	B one box	C fleet	D hybrid	Decision (B-as-D)
Isolates the spike off owned iron	❌	❌ (spike lives inside)	⚠️ CPU only, egress still on uplink	✅	✅ by elasticity, when measured
One consistency + backup domain for state	⚠️ shared w/ estate	✅	❌ fragmented source of truth	✅	✅
Ops-service count / small team	✅ (no new iron)	✅	❌ 3× surface	◑ (2 rental integrations)	✅ (one box now, edge later)
No owned idle GPU / egress	✅	⚠️ tempts owning both	❌ owned GPU idles	✅	✅
Fault isolation of stateful core (SPOF)	❌ estate fate-share	❌ everything, one blast radius	◑	◑ (edge isolated, core still single)	◑ → standby when justified
Reversibility (grow by repoint)	❌	✅	❌ calcifies	✅	✅ by construction
Observable saturation (small footprint)	⚠️	✅ fully probe-able	❌ N× correlation	⚠️ rented edge opaque	✅ + client-side SLI on edge
Source-of-truth survival (off-box)	❌	⚠️ if drilled	⚠️	✅	✅ gated before first write

8. Consequences

New locked invariants (promote into the foundation spec §7, continuing ADR-001's I9/I10):

• I11 — All durable state lives in one owned consistency + backup domain; spiky stateless workloads run off-box on rented/burstable capacity and hold no source of truth. Isolation is bought by elasticity (rent and release the spike), never by owning a bigger box and never by a workload-split fleet. The fault line is stateful-vs-spiky, never workload-vs-workload. (load-shape + data lens — the topology's spine.)
• I12 — The source of truth is never in the same failure domain as its only durable copy. A WAL-consistent, off-box, restore-tested backup path gates the first member write. (Kleppmann — "one box" ≠ "one disk"; an untested backup of the log is a wish.)
• I13 — Media and any relocatable byte-store are addressed by content-hash / URL-indirection through a storage abstraction, so the origin moves without rewriting references. (the one addressing decision that is a migration, not a repoint, if fumbled.)

New required evals (the topology is only as trustworthy as these):

• P0 — Restore-the-source-of-truth drill, fail-closed. WAL-consistent (not naïve file-copy) snapshot of the log + relational store to a separate off-box target, plus a scheduled, logged restore-to-scratch drill. Gate before the first member write — sibling to ADR-001's P0 tier-leak eval. No restored backup = the moat is a wish that fails silently at restore time.
• P0 — Synthetic live-event load test, active-benchmarked. Ramp concurrent players until a named resource saturates (NIC bandwidth, proxy connection table, disk segment reads), watching U/S/E while the load runs. Sizes the rented delivery tier against a measured peak, and sets the written repoint threshold in I11. Gate before the first real live event with paying members.
• P1 — Client-side video SLI. Player start-time, rebuffer ratio, segment-fetch latency measured from the member's browser — so the rented delivery tier is observed by its actual SLI (does video start and stay smooth) even where vendor egress metrics go dark.
• P1 — Outbox lag / queue depth as a first-class saturation metric. The surface→index outbox (ADR-001) is at-least-once; if it backs up under live-event write load, the AI home goes stale silently. Alert on outbox lag, not just node CPU.
• P1 — Inference-call data-residency eval (hand-off to E3/E6 threat model, see dissent): prove the chosen inference path (no-retention API vs. ephemeral rented GPU vs. owned) satisfies I5 before any private, tier-gated member content transits it.

E1's first task is instrumentation + a load test, not a build: wire U/S/E (per store, proxy, NIC, and — if present — GPU queue depth) and RED at every service boundary into the estate telemetry backend, then run the synthetic live-event load test to size the rented tier against a measured peak. Only then finalize the box shape. Forensic instrumentation added mid-incident is archaeology during a customer outage.

Estate seams (what we actually reuse — from the scout):

• Deploy doctrine (REUSE): the hook-injectable deploy wrapper (capture → deploy → fail-closed smoke → auto-revert, single-flight lock, bounded one-revert-then-HOLD) is the platform's CI deploy gate. The push-to-deploy pipeline already staged for this host (deploy config + force-command script + live healthcheck fix) is a half-installed E1 artifact — finish it, don't rebuild it. Host-hardening/secrets/ingress/backup/observability/verify/incident runbooks are the operating manual.
• Agent control surface (ADAPT, post-build): dry-run-default + host-registry-as-allowlist + write-with-.bak — the safety envelope for any agent that later touches the platform host.
• Ops/health console (STRETCH, later): daemon-owns-all-privileged-shelling + thin read-only clients + server-side redaction — the right skeleton if E1 later wants an infra-health console; today it watches task state, not iron.

9. Preserved dissent (do not resolve away)

• Renting inference vs. the privacy moat (load-shape lens's self-dissent): invariant I5 says never hand core data to an un-owned vendor, and the moat is AI over private, vetted, tier-gated member content — which D ships to a third-party inference endpoint as retrieved context on every call. A zero-retention contract is a promise, not a control. If contractual/audited no-retention (a) and ephemeral-rented-dedicated-GPU-running-a-self-hosted-model (b) do not survive the E3 threat model, the answer shifts toward an owned GPU sooner than this ranking admits. Resolve explicitly in E6/E3 (P1 data-residency eval); do not hand-wave. The topology decision here is deliberately compatible with all three outcomes — the AI index stays owned regardless; only the inference compute's residency is open.
• The rented edge is the part I can't observe (Gregg's self-dissent): the moment video delivery and inference leave the box, USE/RED coverage goes dark exactly where load is heaviest, and you're trusting a vendor dashboard for the resource most likely to fail during a live event. The rebuttal — own the client-side signal (P1 video SLI) — is a mitigation, not full parity. A single larger fully-probe-able owned box (B without the rented edge) is a legitimate posture for a team that values "one thing I can completely see" over "two tiers, half opaque."
• One box is a SPOF for everything (Torvalds' dissent): a kernel panic, full disk, bad deploy, or OOM cascade (this estate has a documented OOM-cascade history) takes learning, community, realtime, AI, and identity down at once — a churn event for a paid "log in and stay" product. The honest answer is not a permanently-staffed fleet; it is backup/restore rigor now (P0) and a warm standby sooner than "when it hurts." Availability has teeth at this scale that raw compute-scale does not — but you answer a SPOF with a tested restore path and one spare, not with a distributed system.
• D can metastasize into C (Kleppmann conceding Torvalds): "own the core + replica + rented GPU + rented CDN" is four integrations a team of two runs at 3am. The survival move is to ship the minimum (one box, spike deferred until measured, replica deferred) — acceptable only because the off-box restore-tested backup is non-negotiable. Defer the replica; never defer the drilled off-box copy of the log.

10. Follow-up (beads to file in Phase B under the E1 epic)

1. Lock invariants I11 + I12 + I13 into the foundation spec §7.
2. P0 restore-the-source-of-truth drill (WAL-consistent + off-box + monthly restore-to-scratch) — gate before the first member write. Sibling to ADR-001's P0 tier-leak eval.
3. P0 synthetic live-event load test (active-benchmarked to a named saturation metric) — sets the I11 spike-repoint threshold; gate before the first live event.
4. Wire U/S/E + RED (stores · proxy · NIC · outbox lag · GPU queue if present) into the estate telemetry backend; add the P1 client-side video SLI.
5. CTO investigation: is E1's "AI inference" new GPU iron or entirely the estate governed-brain + agent-governance plane? Measure the footprint before provisioning any GPU-hour.
6. Finish the staged push-to-deploy pipeline for this host on the reused deploy-wrapper (fail-closed smoke + auto-revert); register the box as a governed estate service-entity with deploy contract, smoke, backup, and owner.
7. Specify the media storage abstraction + content-hash/URL-indirection (I13) so the origin→CDN move is a repoint.
8. Write the spike-repoint runbook: the measured threshold (from #3) that promotes video delivery / inference off-box, and the exact repoint steps.
9. Hand the inference data-residency question (P1 eval) to the E3/E6 threat model; keep the AI index owned regardless of outcome.

---

Panel transcript basis: estate-scout + load-shape/observability/simplicity/data-durability lenses, 2026-07-22. Follows the ADR-001 template.