Waldo · Agent Harness · System Design
The Lead Engineer Map
One page that holds the whole system: the mental model, the architecture, the agent loop, how data flows across Cloudflare DO / R2 / Supabase, the authentication boundaries, and where the build actually stands in Linear.
First principles
1 · The Mental Model
Waldo is not an app with an AI feature. It is a persistent per-user agent process — one Durable Object per human — whose first context stream happens to be the human body.
Eight invariants. If a design proposal violates one of these, it is wrong before any code review:
INVARIANT 01
The DO is the agent
Not a function that calls an LLM — a process that wakes on alarms and events, assembles context, runs a bounded loop, delivers, and hibernates. There is exactly one brain per user and it lives in the DO (ADR-0052). Edge Functions never call an LLM.
INVARIANT 02
98.4% deterministic, 1.6% model
The harness is deterministic infrastructure: hooks, ACLs, sanitisers, schedulers, math. If a deterministic function can do the job (CRS, pre-filters, topic rules), it must. The LLM is reserved for judgment.
INVARIANT 03
One writer per truth
Supabase owns raw health + identity + consent. DO SQLite owns agent state + memory. R2 owns cold archive + workspace files. Never two writers, never duplicated authority. Raw biometrics never enter DO or R2.
INVARIANT 04
The prompt is compiled, not written
Context is assembled fresh every wake from typed layers (REASONS: 7 conceptual / 10 physical). Memory is consulted deterministically (recall-before-act) — never left to the model to remember to ask.
INVARIANT 05
Security lives at seams, not in prompts
The model may propose anything; handlers refuse. Per-trigger tool ACLs, the autonomy gate, canary tokens, taint gates and the sanitiser are code-owned boundaries. A prompt instruction is never a security control.
INVARIANT 06
Contract-first
@pin4sf/waldo-types compiles before backend code exists. Tool counts, trigger names, card shapes — all derive from the type union, never from prose. Zod parses every boundary.
INVARIANT 07
Trust resets, memory persists
Every wake is a fresh permission slate (session trust reset) — yesterday's authorization never carries forward. But the brain (halls, episodes, skills) is durable. Memory is information, never authorization.
INVARIANT 08
The cost knob is the escalation rate
The primary model is swappable config behind the AI Gateway; what moves cost an order of magnitude is the escalation rate to the reasoning tier. ~$0.30/user/day blended; pre-filters and KAIROS skips are cost controls, not niceties.
The map
mental model — mermaid mindmap
mindmap root((Waldo
one agent per user)) Agent harness DO runtime, hibernates Run journal and outbox Hook pipeline, 9 events ReAct loop, max 3 iters Typed tools + per-trigger ACL Model routing via AI Gateway Body as context Health pipeline to CRS Form Zones energized to depleted Narrative context in every prompt Calendar, tasks, comms signals Memory 5 typed halls Episodes with FTS5 search Scribe inbox merge Recall before act Dreaming Mode at 2am Interface layer Brief, Fetch, Spots, Chat Patrol audit, Handoff EPA Telegram rich two-way APNs and FCM push Contracts waldo-types barrels Zod at every boundary Semver pinned, no drift Trust and safety Session trust reset Sanitiser and canaries Autonomy gate L1 L2 L3 Taint gate, GDPR erasure Economics Escalation rate is the knob Pre-filter and KAIROS skips Tier-aware spend cap, open
System design
2 · System Architecture
Three repos (waldo-app · waldo-backend · waldo-types), two trust zones (Supabase, Cloudflare), one shared contract. The app talks to EFs over REST; the Worker routes every agent event to that user's DO; the DO reads derived health from Supabase and delivers through channel adapters.
architecture — trust zones & modules
flowchart TB
subgraph SURF["USER SURFACES"]
APP["iOS app
Expo SDK 53"]
TG["Telegram
rich two-way"]
PUSH["APNs / FCM"]
end
subgraph SB["SUPABASE TRUST ZONE — deterministic only, no LLM"]
EF["13 Edge Functions
JWT → rate-limit → Zod"]
PG[("Postgres + RLS
raw health · identity · consent
chat mirror · agent_logs")]
BI["build-intelligence EF
zero-LLM CRS math"]
end
subgraph CF["CLOUDFLARE TRUST ZONE"]
WK["Worker router"]
subgraph DO["WaldoAgent DO — one per user"]
RUN["runtime
run journal + outbox"]
HK["hook pipeline
9 lifecycle events"]
CTX["context module
REASONS prompt builder"]
TL["tool dispatcher
per-trigger ACL"]
MEM[("DO SQLite
halls · episodes · inbox
runs · outbox · budgets")]
SAFE["safety gates
medical · canary · taint"]
end
R2[("R2
workspace files
cold JSONL archive")]
GW["CF AI Gateway
single LLM control plane"]
end
LLM["Gemma 4 26B primary
Sonnet 4.6 escalation ~5%"]
APP -- "REST + JWT" --> EF
EF --> PG
PG --> BI
BI --> PG
APP -- "chat · approvals" --> WK
TG -- "webhooks" --> WK
WK -- "route by user_id" --> DO
DO -- "per-user RLS JWT
derived health only" --> PG
CTX --- MEM
MEM <--> R2
RUN --> GW
GW --> LLM
DO -- "outbox flush" --> TG
DO -- "outbox flush" --> PUSH
DO -- "cards · thread state" --> APP
Deep modules — the interfaces that matter
Each module is deep: a small interface hiding a large implementation. Callers (and tests) know the seam, never the internals.
| Interface (the seam) | What it hides |
|---|---|
tick(runId) → RunState | The whole invocation state machine: run journal, outbox, ReAct iterations, crash-resume after eviction (ADR-0054) |
computeCRS(snapshot) → CrsResult | SAFTE-FAST weights, time-of-day normalization, multi-source HRV reconciliation, pillar drag — pure math, zero LLM |
buildPrompt(trigger, ctx) → string | 7 conceptual REASONS layers as 10 physical layers, recall block, R2 workspace files, provider shaping, channel hints, sandwich defense |
dispatchTool(call, ctx) | Zod parse → ACL check → autonomy gate → taint gate → adapter call → result sanitise → typed error taxonomy |
retrieve(args) / recall(ctx, hint?) | Hall selection per trigger, FTS5 BM25 + temporal + RRF fusion, CARA/ACT-R weighting, hot-DO ∪ cold-R2 federation — the agent never sees storage tiers |
submitMemoryIntent(intent) | Sanitise-at-write, inbox staging, nightly Scribe merge with pattern_id supersedence, bi-temporal history |
enqueueDelivery(intent) → OutboxEntry | Idempotency, push budget, collapse-id stacking, channel persona slicing, retry semantics |
sanitise(input, destination) | 5 ordered checks — destination-aware: internal contexts keep derived scores, external surfaces get zone language (ADR-0024) |
evaluateTrace(trace) → TraceEval | LLM-judge rubrics, WIS components, KeepRate, replay fixtures — generation never grades itself |
deleteUserEverywhere(userId) | GDPR erasure across Supabase + DO SQLite + R2 + observability stores + channel residuals (ADR-0055) |
Workflow
3 · The Agent Loop
Every invocation — a 7am Brief, a 15-minute Patrol pass, a Telegram message — runs the same spine. The only variation per trigger is which tools are allowed, which memory halls load, and which model route fires.
3a · One invocation, end to end
invocation workflow — mermaid sequence
sequenceDiagram autonumber participant T as Trigger
(alarm · webhook · user msg) participant W as Worker router participant H as Hook pipeline participant J as Run journal participant M as Memory participant P as Prompt builder participant G as AI Gateway → LLM participant D as Tool dispatcher participant Q as Quality gates participant O as Outbox participant C as Channels T->>W: event arrives W->>H: route to this user's DO H->>H: session trust reset — fresh ACL, fresh canaries, rate limits H->>H: emergency detection — BEFORE any LLM call H->>J: startRun / tick(runId) — resumes if DO was evicted H->>H: pre-filter — calm + high Form → template, skip LLM entirely M->>P: recall(ctx) — halls + episodes + evolutions, fenced P->>P: REASONS prompt — 10 layers, provider-shaped, under 10K tokens loop ReAct — max 3 iters (7 for chat) P->>G: generate G->>D: tool calls D->>D: Zod parse → trigger ACL → autonomy gate → taint gate D->>G: sanitised JSON results (capped, fenced) end G->>Q: candidate output Q->>Q: medical scrub → hallucination guard → canary check Q->>O: side-effects committed in SAME txn as state advance O->>C: idempotent flush — Telegram / APNs / in-app O->>J: state → DELIVERED → DONE J->>M: write episode · trace · memory intents · WIS signals
TRIGGER TAXONOMY (ADR-0015)
brief (+ variant morning/midday/evening/event) · fetch_alert · patrol · handoff_explore/plan/act/replan · intervention · user_message · dreaming_mode · pre_activity_spot (v0.2.0). The trigger decides everything downstream.
ITERATION BUDGETS
Automated triggers: 3 ReAct iterations max. user_message/Handoff: 7. Diminishing-returns guard: 3 tiny outputs in a row → hard fallback. Concurrent tool calls run via Promise.all.
4-LEVEL FALLBACK
L1 full-context primary → L2 reduced-context primary → L3 template with real data (no LLM) → L4 silent + retry next cycle. Circuit breaker: 3 failures → OPEN (jump to L3) · 2 successes → CLOSED. The user never sees an error.
3b · Durability — the run journal state machine (ADR-0054)
Cloudflare may evict a DO mid-loop. The answer is not "hope": every step is one committed DO SQLite transaction, and every side-effect rides the outbox with an idempotency key — so a crashed Brief resumes instead of double-sending.
run lifecycle — mermaid state diagram
stateDiagram-v2
[*] --> PENDING : startRun(trigger, variant, nonce)
PENDING --> CONTEXT_BUILT : recall + prompt committed
CONTEXT_BUILT --> LLM_CALLED : model response committed
LLM_CALLED --> TOOLS_DONE : tool batch committed
TOOLS_DONE --> GATED : safety gates passed
GATED --> DELIVERED : outbox flushed (idempotent)
DELIVERED --> DONE : episode + trace written
LLM_CALLED --> FAILED : retries exhausted → fallback chain
FAILED --> [*]
DONE --> [*]
note right of CONTEXT_BUILT
Each arrow = one DO SQLite transaction.
DO evicted anywhere? Next wake calls
tick(runId) and resumes from the last
committed state. The outbox dedupes,
so resume never double-fires a push.
Key = hash(user, trigger, variant, run_nonce)
end note
THE 9 HOOK EVENTS (ADR-0032 · HEY-12)
OnInvocationStart (auth → rate-limit → canary gen → session reset → audit-open) · PrePromptBuild (skill loader, recall) · PostPromptBuild (size check, canary inject) · PreLLMCall (cost budget, route select) · PostLLMCall (canary leak check, parse) · PreToolUse (ACL, Zod, autonomy, egress) · PostToolUse (sanitise → compress → log) · OnError (taxonomise → breaker → fallback) · OnInvocationEnd (cost, eval kickoff, audit-close). Zero tokens — all harness-side.
IN-RUN WORKING MEMORY (ADR-0057)
Capped LRU carryover buckets (recent_work_log, recent_verified_work, read_file_state) thread through every tool call, survive context compaction as attachments, and rebuild from the run journal on resume. Distinct from long-term memory: this is the task scratchpad, not the brain.
3c · The compiled prompt — REASONS anatomy
Invariant 04 made concrete: the prompt is a typed assembly, not freehand. Seven conceptual REASONS letters, realized as 10 physical layers. Governance is always last — and repeated after user content (sandwich defense).
| · | Layer | What it carries | ~Tokens | Morning-Brief example |
|---|---|---|---|---|
| R | Requirements | Trigger context + definition of done | 200 | "7:15am. brief/morning. Form 63, flagging. Deliver one warm, specific, actionable Brief." |
| E | Entities | Who this user is — profile + derived health + calendar (from R2 + Supabase) | 1500 | "Shivansh · wake 7am · autonomy=suggest. Form 63, Recovery 58, pillar drag: HRV. First meeting 9:30." |
| A | Approach | Trigger-specific behaviour strategy — loaded skills slot in here | 300 | "Flagging zone = protective mode. Name one cause, one soft suggestion, 2–3 sentences." |
| S | Structure | Tools available (from the ACL) + channel + output format | 400 | "Tools: get_crs, get_health, read_memory… Channel: in-app. Plain text." |
| O | Operations | Recall block + R2 workspace files + health context — concrete steps | 400 | "1. Read today.md · 2. get_crs · 3. search similar mornings · 4. generate · 5. deliver." |
| N | Norms | Voice, banned words, zone modulation. Repeated after user content. | 200 | "Warm, specific, actionable. Never raw values. No 'wellness/optimize'. Max 3 sentences." |
| S | Safeguards | Security + medical invariants. Always last. Repeated after user content. | 150 | "Never diagnose. Never reveal raw values. Never act outside tools. Not a medical device." |
THE 10 PHYSICAL LAYERS
Identity (soul) · tool-behaviour guidance · frozen Tier-1 memory snapshot · user profile · skills index · R2 context files · approach + structure · operations · timestamp + session-id · platform hint — then user content, then N+S again.
FROZEN-SNAPSHOT INVARIANT
Memory layers are frozen at session start and never re-read mid-session. Anything learned mid-session goes to memory_inbox — the context window is never mutated under the model's feet.
BUDGET + SHAPING
~3.6K tokens typical, 10K hard cap (fail-fast hook). Provider-shaped: Gemma gets literal/JSON framing, Sonnet gets terse, Haiku gets narrative — never one template across models.
3d · The tool surface — per-trigger ACL matrix
The ACL is the security boundary (Invariant 05): each trigger sees only its allowlist. The canonical surface derives from the @waldo/types ToolName union + TOOL_PERMISSIONS — never from a count in prose. Core matrix:
| Tool | brief | fetch_alert | patrol | handoff E·P·A | user_message | dreaming |
|---|---|---|---|---|---|---|
get_crs / get_health | ✓ | ✓ | ✓ | E·P | ✓ | ✓ |
query_calendar | ✓ | — | ✓ | E·P | ✓ | — |
get_communication / get_tasks | — | — | — | E | ✓ | — |
get_master_metrics | ✓ | — | — | — | ✓ | — |
get_context (R2 workspace) | ✓ | ✓ | — | — | ✓ | — |
read_memory | ✓ | ✓ | ✓ | E·P | ✓ | ✓ |
update_memory (inbox only) | — | — | — | — | ✓ | ✓ |
search_episodes | ✓ | — | ✓ | — | ✓ | ✓ |
propose_action | ✓ | — | — | P | ✓ | — |
execute_action | — | — | — | A only ⚠ | ✓ | — |
send_message | ✓ | ✓ | ✓ | P·A | ✓ | — |
web_search ★ / read_document ★ | — | — | — | E | ✓ | — |
call_mcp_tool ★ | — | — | — | — | ✓ | — |
execute_code ◆ / browser_use ◆ / analyze_image ◆ | — | — | — | — | — | — |
★ V1 general-agent trio (ADR-0049) — ships with the taint gate. ◆ Typed but disabled until Phase 3 (ADR-0050). Copilot writes (write_task · update_task · draft_document · draft_email · propose_schedule · write_sheet_cell) live in handoff_act + user_message, all behind the autonomy gate — connector_write always confirms. search_connector adds to brief/patrol/handoff-explore/chat. Threading tools (create_thread · delete/restore_message · archive_thread · update_thread_topics) are chat-only. The intervention trigger gets reads + update_task + propose_action — and never send_message (interventions don't push).
3e · Skills — the 15 bundled recipes (ADR-0022)
Tools are primitives; skills are recipes — markdown files composing tools + a reasoning template + a trigger condition + an effectiveness score. Every user gets these 15 on day one. A skill can never exceed its trigger's ACL (required_tools ⊆ TOOL_PERMISSIONS) — recipes don't grant privileges.
| # | Skill | Category | Fires on | What it does |
|---|---|---|---|---|
| 1 | morning-brief | Lifecycle | brief · morning | The day-opening read: Recovery, what was handled overnight, day context |
| 2 | midday-brief | Lifecycle | brief · midday | Form trend since morning, Weight build-up, what's ahead |
| 3 | evening-brief (The Close) | Lifecycle | brief · evening | Day summary, Form trajectory, wind-down recommendation |
| 4 | event-brief | Lifecycle | brief · event | Fires on significant intraday change (Form −15, HR spike) — heightened attentiveness, plan is dynamic |
| 5 | fetch-alert | Lifecycle | fetch_alert | The acute stress alert (shadow-mode in V1) |
| 6 | intervention | Lifecycle | intervention | Overload escalation: Form floor + Load ceiling + complex task + 3 declining Patrol entries (ADR-0020) |
| 7 | patrol-pass | Lifecycle | patrol | The every-15-min background observation cycle (ADR-0017) |
| 8 | prep-for-event | Copilot | brief · patrol · chat | Pre-meeting brief: attendees, last meeting's open items, connector context |
| 9 | compose-today | Copilot | brief · chat | "What's on my plate today" — schedule + tasks with Form-aware ordering |
| 10 | summarise-meeting | Copilot | chat · dreaming | Transcript → summary + decisions + action items → tasks |
| 11 | recovery-day | Copilot | brief · patrol | Auto-fires when Form < 50 sustained: rearranges the day for recovery, UI density drops |
| 12 | focus-block-prep | Copilot | patrol · chat | Loads working context (docs, issues, tasks) as a Focus Window starts |
| 13 | weekly-review | Copilot | brief · chat · dreaming | Friday/Sunday week roll-up document |
| 14 | catch-up | Copilot | chat · brief | Returning after a 48h+ gap: "here's what changed while you were away" |
| 15 | permission-slip | Social | chat | Generates a socially-credible boundary message ("Waldo says I need to take it easy") — the viral mechanic |
FOUR PROVENANCES, ONE IDENTITY BOUNDARY
system (git, immutable, identity-locked — the 15 above) · connector (auto-load on connect: linear-sprint-summary, notion-spec-draft, gmail-priority-triage, spotify-mood-check…) · user ("make that a routine" → R2 /skills/user/) · agent_authored (Dreaming Mode spots a tool-sequence repeated 5+ times in 14 days → proposes via the autonomy gate, lives 7 days as provisional). Anything touching medical guidance, CRS, safety rules, voice, or the lifecycle is identity-locked — no runtime edit, ever.
EFFECTIVENESS — SKILLS EARN THEIR SLOT
Score = 0.6 × rule-based signals (thumbs, completion, dismissal, undo) + 0.4 × nightly LLM-judge. Below 0.40 three times → auto-revert / review. Above 0.80 ten times → preferred routine. Loader takes top-K per trigger (Brief 5 · chat 8). Agent-authoring caps: ≤2 new skills/week, never on safety-critical triggers, never execute_code without explicit user approval. Note: the learning loop is Phase-G; the pre-build scope register proposes V1 ships these as static markdown first.
Data architecture
4 · Memory & Storage
Three stores, three jobs: Supabase is the system of record, DO SQLite is the hot working brain, R2 is the cold archive plus the agent's filing cabinet. Nothing is ever in two places with two writers.
4a · Who owns what
SUPABASE POSTGRES · ap-south-1 · system of record
Raw truth, queryable, regulated
Holds: health_daily (raw device readings — the only place raw HRV/HR ever persists) · crs_scores (derived) · users / user_consents (Art. 9 consent before first health write) · oauth_tokens (Vault-encrypted) · chat_threads/messages mirror · patrol_entries · agent_logs · spots · notification_log.
Writers: EFs (sync, intelligence) + DO (audit/chat/logs via scoped writes).
Guard: RLS on every table, day 0.
DO SQLITE · per user · hot working set
The agent's brain, sub-ms reads
Holds: memory_blocks (5 typed halls, bi-temporal, pattern_id) · episodes + FTS5 · memory_inbox (Scribe staging) · runs/outbox (ADR-0054) · thread_topic_index · daily_push_budget · goals · commitments · handoff_state · interventions · skills · agent_evolutions.
Writer: the DO alone — single-writer by construction.
Guard: AuditedDB blocks UPDATE/DELETE on append-only tables. 10 GB soft cap → ADR-0056 compaction.
R2 · per-user prefix · $0 egress
Cold archive + workspace
Holds: waldo-workspace/{user}/ — today.md · baselines.md · patterns.md · goals.md · orchestration.md · skills/ (pre-compiled context, written nightly) · waldo-episodes/{user}/y/m/episodes.jsonl (JSONL archive >90d) · GDPR exports (signed URL, 24h).
Writer: the DO via a 5-function mount (read / write / list / commit / discard).
Guard: archive-don't-delete; only GDPR erasure removes rows.
R2 as the agent's LLM wiki — yes, by design
Can the agent treat R2 as a markdown knowledge base — an LLM wiki of facts plus its working files? Yes, in two levels — with three non-negotiable constraints.
LEVEL 1 — SHIPPING IN V1 (the workspace)
R2 already is the agent's filing cabinet: today.md · baselines.md · patterns.md · goals.md · orchestration.md · skills/*.md — written nightly by Dreaming Mode through the WorkspaceMount (read / write / list / commit / discard) and injected into every prompt. Files-as-interop: any model, any provider, reads the same markdown — no lock-in.
LEVEL 2 — THE FULL WIKI (planned, roadmap M3 · HEY-76)
Per-hall markdown pages with a Compiled Truth paragraph + append-only Evidence Trail + backlinks — exactly the pattern this vault uses on itself. Specced in the agent-intelligence design (6 hall files, INGEST/QUERY/LINT cycle); ticketed as HEY-76 (memory-tree cascade-seal + Obsidian mirror). The wiki becomes the user-readable, exportable view of what Waldo knows.
CONSTRAINT 1 — COMPILED VIEW, NOT SOURCE OF TRUTH
Typed facts live canonically in DO SQLite memory_blocks (confidence, bi-temporal validity, pattern_id, per-hall ACL). The R2 wiki is generated from the halls — a build artifact. If they ever disagree, the halls win. Two writers to one truth breaks Invariant 03.
CONSTRAINT 2 — NO RAW HEALTH, EVER
The destination-aware sanitiser runs before every R2 write: derived zones and narratives go in, raw HRV/HR/sleep numbers never do. R2 prefixes are per-user, but it is still outside the health system-of-record.
CONSTRAINT 3 — R2 HAS NO SEARCH
No FTS, no SQL — R2 is read-by-path. The design absorbs this: the common case is pre-compiled (the agent just opens today.md), and the long tail goes through retrieve() where keyword search lives in DO FTS5. Use R2 for "read the file I know," never "find me something."
The 5-tier memory stack
| Tier | Where | What | Budget / lifecycle |
|---|---|---|---|
| Tier 0 — context window | volatile | REASONS prompt assembled fresh each wake | ~10K tokens, never stored |
Tier 1 — memory_blocks | DO SQLite | 5 halls: facts · events · discoveries · preferences · advice | ≤200 tokens always loaded · CARA confidence · ACT-R decay |
Tier 2 — episodes | DO SQLite | Per-session diary entries, FTS5-indexed | 0–90 days hot, then R2 |
Tier 3 — skills / procedures | DO SQLite + R2 | 15 bundled + connector + user/agent-authored skills | effectiveness-scored, top-K loaded per trigger |
| Tier 4 — cold archive | R2 | JSONL episodes >90d + memory-history chains | forever (GDPR erasure only) · reachable via retrieve() |
4b · Health dataflow — raw stays put, zones travel
health pipeline — device → supabase → derived context
flowchart LR
subgraph DEV["DEVICE — private"]
AW["Apple Watch sensors"] --> HKS["HealthKit store"]
HKS -- "Background Delivery wake" --> NM["Swift module
anchor-based incremental"]
NM --> OPS[("op-sqlite + SQLCipher
intraday raw stays here")]
end
subgraph SBZ["SUPABASE — system of record"]
EF1["sync EF — JWT
unit normalize · anomaly bounds"]
HD[("health_daily
1 row / user / day")]
BIX["build-intelligence
zero-LLM CRS math"]
CS[("crs_scores
Form · zone · pillars")]
end
subgraph DOZ["DO — derived only"]
BC["body-context module"]
NCX["narrative context block
zones + trends, NEVER raw values"]
end
NM -- "JWT POST" --> EF1
WHX["WHOOP / Oura cloud"] -- "HMAC webhook" --> EF1
EF1 --> HD --> BIX --> CS
CS -- "per-user RLS JWT" --> BC --> NCX
Rule of the wall: raw sensor values (HRV ms, HR bpm, sleep minutes) exist only on-device and in health_daily. The DO reads derived scores; the sanitiser is destination-aware — internal contexts keep the CRS number, anything user-facing or external gets zone language ("flagging", "rough night"). Multi-device conflicts resolve by hrv_confidence per metric.
THE LOCKED FORMULA (ADR-0011 · SAFTE-FAST GROUNDED · 856-DAY VALIDATED · ZERO LLM)
Form = Sleep × 0.50 + HRV (CASS) × 0.35 + Circadian × 0.075 + Motion × 0.075
Recovery = Sleep × 0.50 + CASS × 0.25 + RHRTS × 0.15 + RRS × 0.10 (backward-looking, fixed at wake). Computed deterministically in build-intelligence — no model anywhere in the health math. The score maps to the persona zone that modulates every message:
80+ energized — challenge mode65–79 steady — balanced, watchful50–64 flagging — protective<50 depleted — minimal, gentle
4c · Memory writes — nothing touches the brain directly
scribe inbox-merge — mermaid sequence
sequenceDiagram participant A as Agent tool call participant S as Sanitiser (5 checks) participant I as memory_inbox participant U as Same-session reads participant SC as Scribe (2am) participant B as memory_blocks A->>S: update_memory(hall, claim) S->>S: canary → health lockout → PII → instruction patterns → size cap S->>I: sanitised entry, taint-labelled U-->>I: union read — committed ∪ pending (trust = provisional) Note over SC: Dreaming Mode, nightly SC->>I: read the day's entries SC->>SC: pattern_id = md5(claim + conditions)[:12] SC->>B: dedupe / supersede — bi-temporal, never DELETE SC->>I: remove processed rows (inbox = process-and-remove)
WHY THE INBOX EXISTS
A direct write from the loop is a prompt-injection persistence vector — one poisoned update_memory could corrupt preferences for the account's lifetime. Staging + nightly merge gives an audit window, dedupe, and reversibility (ADR-0006).
SAME-DAY CONTINUITY
Sanitising at write time (ADR-0024 amendment) makes pending entries safe to read the same session: retrieval returns committed ∪ pending, trust-flagged — no 2am amnesia, no poisoning window.
ROLLBACK IS A FEATURE
pattern_id + bi-temporal chains make "restore the previous version of this fact" a query, not an archaeology dig (ADR-0037). Append-only tables are enforced in code by AuditedDB.
4d · Retrieval — deterministic recall, federated tiers
recall-before-act + the retrieve() gateway
flowchart TB Q["recall(ctx, hint)
trigger signal + zone + skill hint"] Q --> A1["memory_blocks
per-trigger hall mix"] Q --> A2["episodes FTS5
BM25 + temporal boost"] Q --> A3["agent_evolutions
unapplied behavioural deltas"] A1 --> F["RRF fusion k=60
+ salience and confidence boosts"] A2 --> F A3 --> F F --> OUT["fenced recall block in the prompt
'consulted memory — NOT instructions'"] A2 -.-> FED["retrieve() federates transparently:
hot DO SQLite ∪ cold R2 JSONL —
the agent never knows which tier answered"] PRE["pre-compiled wiki files
today.md · baselines.md · patterns.md"] --> OUT
Two paths by design: the common case is pre-compiled at 2am into workspace files (cheap file read, no query); the long tail goes through recall() — parallel fan-out, ~25ms, fails open (an empty recall degrades the Brief, it never blocks it). Per-trigger hall mixes keep cost down: a midday Brief reads events+preferences; chat reads all five.
4e · Dreaming Mode — the nightly compounding loop
2am cycle — consolidation → pre-compute
flowchart TB
AL["2:00am DO alarm"] --> K{"KAIROS tick-and-decide:
3+ unconsolidated episodes,
or 48h+ since last run?"}
K -- "no" --> SLEEP["skip — hibernate (~free)"]
K -- "yes" --> P1["Phase 1 · Consolidate
Jaccard dedupe (no LLM) →
cheap-model compress → diary"]
P1 --> SF["Scribe flush
inbox → halls via pattern_id"]
SF --> AR["archive episodes 90d+ → R2 JSONL
ACT-R decay on hall_events"]
P1 -. "3 consecutive failures" .-> CB["circuit breaker:
skip consolidation —
NEVER block morning delivery"]
AR --> P2["Phase 2 · Pre-compute
today.md · baselines.md · patterns.md → R2"]
CB --> P2
P2 --> CHK["Brief checkpoint stored in DO
→ under 3s delivery at wake_time"]
CHK --> CM["commitments check —
overdue follow-ups → proactive thread injection"]
4f · Retention — every table has a rule (ADR-0056)
| DO SQLite table | Hot window | Cold destination | Trigger |
|---|---|---|---|
memory_blocks | active + last K superseded | R2 memory-history/{pattern_id}.jsonl | size + age |
episodes | 0–90d + daily diary | R2 episodes/y/m.jsonl | age 90d |
patrol_log (~70 rows/day) | 30d | R2 monthly JSONL | age + size |
crs_history | rolling 30d baseline window | R2 (re-derivable from Supabase) | age 30d |
runs / outbox | active + 7d | R2 run-journal archive | on-complete + age |
memory_inbox · budgets · rate windows | transient | n/a — consumed or auto-expire | process-and-remove |
Two compaction triggers: the nightly time-based sweep and a do_bytes size watermark (60% of the 10 GB cap → evict coldest-first; 80% → alert). Row-DELETE is allowed in exactly two places: self-clearing transients and GDPR erasure. Storage compaction is invisible to the agent — moving a row DO→R2 only changes which leg of retrieve() finds it.
4g · The math under the hood
Four algorithms — implemented in plain TypeScript inside the DO, zero external dependencies — are what make the memory smarter than a key-value store.
CARA — CONFIDENCE THAT LISTENS
+0.1 when the user confirms a remembered pattern · −0.2 when they contradict it · below 0.5 the block stops loading into context. Why: discoveries can be wrong — two contradictions and a bad pattern silences itself instead of poisoning every future prompt.
ACT-R — FORGETTING AS A FEATURE
activation = ln(Σ tᵢ⁻⁰·⁵), threshold θ = −2.0. Decay by age: 1d ≈ 1.0 · 7d ≈ 0.38 · 30d ≈ 0.18 · 90d ≈ 0.10. Why: a three-month-old diary entry shouldn't compete with yesterday's for the 200-token always-loaded budget — it fades below threshold (archived, never deleted).
BM25 + TEMPORAL → RRF — RETRIEVAL FUSION
BM25 (k1=1.2, b=0.75) for keywords · temporal boost ×1.5 recent / ×0.3 idle · fused by score = Σ 1/(60 + rank) · post-fusion ×1.15 if citation_rate > 0.3. Why: combines "what matches" with "what's fresh" and "what Waldo actually uses" — no vector DB needed for V1.
BI-TEMPORAL — TIME-TRAVEL FOR FACTS
valid_from / valid_to / superseded_by on every block. An update invalidates and inserts — never deletes. Why: rollback of poisoned memory, point-in-time queries ("what was my baseline in March?"), and honest GDPR exports all come free from one schema decision.
Identity & boundaries
5 · Authentication & Security Model
Five credential domains, none interchangeable: the user's JWT, the EF-only service-role key, the DO's per-user RLS JWT, webhook signatures, and connector OAuth in Vault. The single most important line: the agent loop never holds a credential that can bypass RLS (ADR-0052).
auth boundaries — who holds which credential
flowchart TB
subgraph USR["USER SPACE"]
APPC["iOS app"]
TGU["Telegram user"]
WHC["WHOOP / Oura cloud"]
RCX["RevenueCat"]
GOOG["Google OAuth consent"]
end
subgraph SBT["SUPABASE TRUST ZONE"]
AUTH["Supabase Auth
issues user JWT"]
EFZ["Edge Functions
validateJWT first 10 lines
rate-limit → Zod → version gate"]
PGZ[("Postgres — RLS day 0")]
VLT[("Vault
OAuth tokens encrypted,
stored as key references")]
WBH["webhooks EF router"]
end
subgraph CFT["CLOUDFLARE TRUST ZONE"]
DOZ["WaldoAgent DO
(NEVER holds service-role)"]
SEC["wrangler secrets
APNs .p8 · bot token · gateway key"]
end
APNS["APNs"]
TGAPI["Telegram Bot API"]
APPC -- "sign-in (SIWA / Google)" --> AUTH
AUTH -- "user JWT" --> APPC
APPC -- "Bearer JWT on every call" --> EFZ
EFZ -- "service-role — EF-ONLY,
RLS bypass confined here" --> PGZ
TGU -- "update" --> WBH
WBH -- "telegram_user_id allowlist —
unknown sender = silent drop.
Linking: one-time deep-link token (10 min)" --> DOZ
WHC -- "HMAC-SHA256 signature" --> WBH
RCX -- "signed webhook" --> WBH
GOOG -- "PKCE flow, state = one-time token" --> VLT
DOZ -- "per-user RLS-enforced JWT
derived health reads only" --> PGZ
DOZ -- "ES256 JWT (1h, per request)" --> APNS
DOZ -- "bot token" --> TGAPI
Inside the loop — defense in depth, in order
| # | Gate | What it stops |
|---|---|---|
| 1 | Emergency detection — regex, before any LLM | Crisis input gets hotlines (iCall India · 988 US), never a model response |
| 2 | Session trust reset — every wake | Yesterday's chat-session permissions bleeding into today's Brief; injection persistence across sessions |
| 3 | Per-trigger tool ACL — deny-first, in the dispatcher | A Brief invocation ever holding execute_action; injection with nowhere to go |
| 4 | Autonomy gate L1/L2/L3 — at the tool handler, not the prompt | Unapproved mutations. connector_write (3rd-party-visible) always confirms — even at L3. Email send requires a user-tap HMAC approval token (5-min expiry) |
| 5 | Taint gate — ships with the V1 general tools | "Ignore previous instructions" in a calendar title flowing into a privileged action; external-derived args force human confirm |
| 6 | Sanitiser — 5 checks at every write/egress seam | Raw health values leaking outward; PII in drafts; instruction patterns entering memory; oversize blobs |
| 7 | Output gates — medical scrub · hallucination guard · canary check | Diagnosis language; numbers the tools never returned; system-prompt leak (canary in output = session terminated) |
| 8 | Egress allowlist + push budget in DO storage | Loop calling unapproved hosts; notification spam surviving crash-retries (3/day, counted durably) |
AUTONOMY GATE
L1 "Just tell me" → observe only, log to Patrol. L2 "Suggest" (default) → inline proposal card, wait for tap. L3 "Move things" → auto-apply + undo, except connector_write which always confirms. Mid-day downgrade triggers the reversibility sweep (ADR-0018).
WHY MEMORY ≠ AUTHORIZATION
A memory block saying "user previously approved bulk send" is information. Today's send still needs the autonomy gate + a fresh confirmation token. Past approval never grants future capability (ADR-0033).
GDPR ERASURE (ADR-0055)
One runbook enumerates every store: Supabase cascade, POST /do/{user}/delete (SQLite + R2 prefix), observability (Langfuse/PostHog/Sentry — hashed IDs, no health payloads), and the Telegram residual (documented, user-instructed). 30-day Apple ceiling. Telegram carrying health-derived text makes this launch-blocking.
Live build state · linear.app/heywaldo · read 2026-06-03
6 · Build State — what Linear actually says
112 issues in the HeyWaldo team, one project (Waldo V1 — Build Sprint), five sprint milestones plus a deliberately parked Phase-G/self-evolution backlog. The repos are greenfield — the contract ships first, then the spine, then skills.
Done 3In Progress 5Todo 57Backlog 46Canceled 1 · ready-for-agent: 79 · needs-info: 6 · external blockers: 2 · by repo: backend 73 · app 12 · brain 7 · types 6
| Milestone | # | What it builds | Spotlight tickets |
|---|---|---|---|
| Sprint 0 — Foundation | 19 | The contract + schemas + security primitives. Nothing else starts until these compile. | HEY-73 ★ types v0.2.0 — IN REVIEW (PR #3) HEY-6 repos HEY-7 types v0.1.0 HEY-9/10 schemas HEY-12 hooks ×9 HEY-13 sanitiser HEY-14/15/16 loader·recall·REASONS HEY-99 spend-cap decision HEY-5 Apple Dev (ext) |
| Sprint 1 — Lifecycle | 12 | Model routing, channels, the 7 lifecycle skills, CRS engine — the first end-to-end Brief. | HEY-17 LLM routing HEY-18 Telegram HEY-19 APNs HEY-20–24 brief/fetch/intervention/patrol skills HEY-102 CRS engine HEY-25–27 threading + injection |
| Sprint 2 — Copilot reads | 11 | App scaffold + HealthKit native module, read-side copilot, channel persona, Dreaming Mode. | HEY-28/29 app + HealthKit HEY-30 search_connector HEY-31 compression HEY-32 persona HEY-37 Dreaming 6-phase HEY-98 general tools (F1) |
| Sprint 3 — Copilot writes | 12 | Doc/Email adapters, mutating tools behind the autonomy gate, Window evaluator, GDPR cascade. | HEY-38/39 Doc+Email adapters HEY-40–43 write tools HEY-48 autonomy gate HEY-46 Window evaluator HEY-101 GDPR delete |
| Sprint 4 — Eval + TestFlight | 8 | SheetProvider, final skills, the verification layer, first internal build. | HEY-50/51 sheets HEY-53 LLM-judge HEY-54 WIS HEY-55 30 golden cases HEY-56 TestFlight |
| Unmilestoned spine | 48 | Runtime core + harness adoptions + the parked self-evolution track. | HEY-72 DO loop shell HEY-110 run journal+outbox HEY-111 eval spine HEY-78 ToolDispatcher ACL HEY-71/74–77/79 adoptions HEY-80–97 Phase-G self-evolution (parked) HEY-63–68 calendar+voice HEY-109 State/session-bus HEY-103/104 ADR-0053 dev-loop |
| Phase 3 — Deferred | 2 | Power tools stay typed-but-disabled until sandbox contracts are proven. | HEY-49 execute_code (ADR-0050) HEY-108 agent-behavior CI evals |
6a · The critical path — what blocks what
dependency spine — from contract to first end-to-end brief
flowchart LR H73["HEY-73 ★
types v0.2.0 spine
IN REVIEW · PR #3"] H112["HEY-112
dep-migration gate
backend+app pin ^0.2.0"] H9["HEY-9
Supabase schema ×17 + RLS"] H10["HEY-10
DO SQLite schema"] H102["HEY-102
CRS engine (zero-LLM)"] H12["HEY-12
hook registry ×9"] H13["HEY-13
Scribe sanitiser"] H15["HEY-15
recall()"] H16["HEY-16
REASONS builder"] H72["HEY-72
DO loop shell"] H110["HEY-110
run journal + outbox"] H78["HEY-78
ToolDispatcher + ACL"] H17["HEY-17
LLMProvider routing"] H99["HEY-99 ⛔ founder call:
tier-aware spend cap"] H18["HEY-18 Telegram
HEY-19 APNs"] SK["HEY-20…24
lifecycle skills"] E2E(["first Brief
end-to-end"]) H73 --> H112 H112 --> H9 H112 --> H10 H9 --> H102 H112 --> H12 --> H13 H10 --> H15 --> H16 H112 --> H72 --> H110 --> H78 H16 --> H17 H99 -. "blocks" .-> H17 H17 --> H18 H78 --> SK H18 --> SK H102 --> SK SK --> E2E
UNIT ECONOMICS (cost-model v2 · 2026-06-02)
Blended ~$0.30/user/day. Pup free (loss-leader, ~$0.03/day) · Pro $14.99/mo ≈ $0.419/day net of Apple's 15% — average Pro costs ~$0.25/day → ~40% net margin · Pro Max $29.99 uncapped. The problem child: power Pro ~$0.53/day = negative at Pro pricing → the ADR-0051 / HEY-99 founder fork. Escalation sensitivity: 5% = baseline · 10% = +$0.10/day · 20% = margin gone — which is why the shadow-eval is load-bearing.
WHAT THE TICKET GRAPH TELLS A LEAD ENGINEER
① Contract-first is enforced, not aspirational — 73 backend tickets sit behind HEY-73/112. ② Security primitives (hooks, sanitiser, ACL dispatcher, autonomy gate) are sequenced before any mutating tool ships. ③ Eval is typed early (HEY-111) even though the judge runs last. ④ 79 of 112 tickets are ready-for-agent — spec'd to run AFK; humans are reserved for OAuth flows, physical-device tests, and decisions. ⑤ The self-evolution ambition (HEY-80–97) is parked where it belongs: behind a working loop and real traces.
THE OPEN RISKS, HONESTLY
① HEY-99 is the only non-technical blocker on the spine — a founder pricing call (power Pro costs ~$0.53/day vs $0.419/day net revenue). ② HEY-5 Apple Developer enrollment gates push + TestFlight. ③ Calendar/voice (HEY-63–68) are backlogged while ADR-0040/41/42 pulled them into Phase 1 — a scheduling contradiction to reconcile. ④ CRS is validated n=1; the Fetch ships shadow-mode until the false-positive bar clears (pre-build register D3). ⑤ Coordination runs through HEY-109's baton protocol — single writer per cluster, cross-review by comment.
Reference
7 · Glossary
The same definitions the hover tooltips use.
Compiled 2026-06-03 · sources: 57-ADR set · WALDO_ARCHITECTURE_OVERVIEW · V1 master/harness/backend plans · ADR-0054/0055/0056/0057 · live Linear read (112 HeyWaldo issues) · non-authoritative visual reference per the vault's source-hierarchy rule.