Keystone Chat — Frontier Dev Stack (Research + Design)
Status: Proposed (for Alex's review — not an accepted ADR) Date: 2026-06-28 Author: Claude (engineer) for Alex (stakeholder)
Decisions locked with Alex (2026-06-28):
- Capability-gated, local-first-when-able. Use a frontier local model when the box can run it, otherwise fall back to Claude (cloud). One harness; the model is auto-selected by detected hardware.
- Local frontier model = Qwen 3.6-27B (the "24 GB model"). Alex will buy or rent aGB GPU for his own box; consumers on ≤GB get the cloud-fallback profile automatically.
- Claude is the cloud brain / escalation ceiling. Models stay interchangeable (North Star), but Claude leads cloud "as of now."
- Recurrent-depth (Ouro) = research front only (SOTA designs).
0. TL;DR
One harness, two model profiles, selected automatically. The frontier of consumer dev is the harness, not the weights — every frontier tool (Claude Code, Cursor, Windsurf, Devin, Cline) is a harness over a cloud (or BYO) model, and theliterature shows up to 10× on coding benchmarks from harness/edit-format changes alone, no model change (Survey). That is the Σ₀ "the loop is everything" bet.
Keystone runs the same harness everywhere and lets the VRAM gate + serving-liveness probe (both already in local-model-registry.js) pick the model:
| Detected box | Local model leads? | Agentic brain | Claude's role |
|---|---|---|---|
| ≥GB GPU (Alex's box, rented GPU) | Yes — Qwen 3.6-27B | Local frontier | Verify-failure escalation (Opus 4.8) |
| ≤GB GPU (typical consumer) | No (support only) | Claude Sonnet 4.6 | Workhorse |
| No GPU / offline | — | Claude (or local offline fallback) | Everything |
Same code path, same KEYSTONE_LOCAL_FIRST verify-gated escalation — only the threshold where local stops leading changes with hardware. No new serving path, no new memory system.
1. What "frontier consumer dev" means in mid-2026
| Tool | Model | Frontier ingredient (all harness) |
|---|---|---|
| Claude Code | cloud | Orchestrator–worker sub-agents (isolated context), 5-tier compaction, permission tiers, MCP/skills/hooks (arXiv 2604.14228) |
| Cursor / Windsurf | cloud | Deep codebase index + flow-aware retrieval |
| Devin | cloud | Long-horizon planning + sandbox + self-verification |
| Aider / Cline | cloud (BYO-key) | Repo-map + search-replace edits + step approval |
| Goose | BYO model | Model-agnostic, on-machine, local-first — closest sibling to Keystone |
The named discipline — harness engineering — has fixed components: tool/execution substrate, state persist + compaction, permission tiers, feedback routing, state-transition verification (Survey); context engineering = compaction + progressive disclosure + just-in-time retrieval. All map 1:1 onto the loop, so chasing the frontier is extension, not sprawl:
| Σ₀ stage | Harness component |
|---|---|
| Observe / Remember | Codebase index, repo-map, JIT retrieval, compaction |
| Reason | Planner + sub-agent delegation (isolated context) |
| Act | Uniform multi-turn tool loop, search-replace edits, sandbox |
| Verify | Test-driven gate, diff-apply re-verify, state-transition checks |
| Converge | Convergence records, escalation + distillation flywheel |
2. Model strategy — capability-gated ladder
Models stay interchangeable; this is a routing policy over the existing provider abstraction + VRAM gate.
Tier L-Frontier — LocalGB (leads when present)
| Model | Params | Ctx | SWE-bench Verified | VRAM (Q4) | Tool-calling |
|---|---|---|---|---|---|
| Qwen 3.6-27B (dense) ⭐ | 27B | 262K | 77.2% | ~17 GB | Native (qwen3_coder) |
| Qwen 3.6-35B-A3B (MoE) | 35B/3B | 262K | 73.4% | ~22 GB | Native; 101.7 tok/s on 3090 |
| Qwen3-Coder-Next (step-up) | 80B/3B | 256K–1M | 71.3% (SWE-rebench Pass@5 64.6%, #1) | ~35–40 GB (48 GB box) | Native |
Source: kilo.ai, InsiderLLM. Pick: Qwen 3.6-27B — dense beats the MoE on raw agentic quality (77.2 vs 73.4), and it matches Claude 4.5 Opus on Terminal-Bench 2.0 (59.3). When it verifies a task locally, you pay zero cloud tokens.
GPU note (Alex's box): ~17 GB Q4 fits a singleGB card — used RTX≈ $700–900, or rent (RunPod/Vast.ai/Lambda, aGB instance runs this comfortably; llama.cpp + speculative decoding measured 2.56× on this exact model/GPU).
Tier C — Cloud brain / ceiling (Claude)
| Role | Model | When |
|---|---|---|
| Workhorse (8 GB profile) | Claude Sonnet 4.6 | Default agentic brain when no local frontier model |
| Ceiling (all profiles) | Claude Opus 4.8 | Verify-failure escalation / hardest multi-file tasks. SWE-bench Pro active leader 69.2% (morphllm) |
Open-weight cloud backups (GLM-5.2, DeepSeek V4-Pro, Kimi K2.6) remain valid drop-ins, but Claude leads now per Alex.
Tier— LocalGB support layer (always, even on theGB box)
OnGB you load one model at a time; on theGB box this work co-exists with headroom. Cheap/frequent/private jobs that never need the frontier:
| Role | Model | VRAM | Loop stage |
|---|---|---|---|
| Resident coder | Qwen2.5-Coder-7B (Q4) | ~5 GB | FIM/autocomplete (Act), cheap edits, intent routing (Reason) |
| Index embeddings | nomic-embed-text / bge-small | <0.5 GB | Codebase index — code never leaves the machine (Remember + privacy) |
Every turn Tierabsorbs is a Claude token unpaid — a margin lever for a subscription/BYO-key product.
Why local can't always be the brain (honest)
A 7–9B model atGB tops out near HumanEval single-file work; its agentic SWE-bench is far below the bar — hence the gate. The 27B frontier model clears the bar but needsGB. The gate is exactly the line between "local leads" and "Claude leads."
Recurrent-depth (Ouro) — research-front
Ouro-1.4B is today's hardcoded Σ₀-native default (toolCalling:false). Genuine research front, but a 1.4B no-tools model can't anchor a tool-driven cockpit. Keep as a research registry entry; the default local lead becomes capability-gated (Qwen 3.6-27B ≥24 GB, else Qwen2.5-Coder-7B support).
3. Tech stack
| Layer | Choice | Status |
|---|---|---|
| Local serving | Ollama (dev) → llama.cpp + spec-decode / vLLM FP8 (24 GB prod) | Ollama wired |
| VRAM detection | nvidia-smi/torch → sets the gate budget (override VRAM_BUDGET_GB) |
Build (today hardcoded 8) |
| Cloud transport | Existing provider router + PCSF → Anthropic first | Have it |
| Tool-call format | Native Claude tool_use; qwen3_coder parser (local) |
Partial |
| Edit format | Search-replace blocks (the 10× format) | Build |
| Index | Local embeddings + ranked repo-map | Build |
| Sandbox | Git worktrees | Have it |
3.1 The capability gate (the one new routing rule)
detect VRAM → selectAvailableChain(taskType) # existing, VRAM-gated + liveness
if best served local model.capabilityScore ≥ FRONTIER_THRESHOLD:
local leads coding/reasoning → verify → on fail, escalate to Claude Opus # KEYSTONE_LOCAL_FIRST
else:
local does Tier-0 support → Claude Sonnet leads agentic → Opus on hard
This is selectChain + capabilityFirst + the existing KEYSTONE_LOCAL_FIRST escalation, with (a) auto-detected VRAM instead of a hardcoded 8, and (b) the Qwen 3.6-27B entry registered with a frontier-tier capabilityScore.
4. The harness — the actual product (5 gaps vs Claude Code)
| # | Build | Stage | Touches | GPU-independent? |
|---|---|---|---|---|
| 1 | Uniform multi-turn agent loop | Act | tool-runner.js, tool-turns.js, stream-chat.js |
✅ testable today vs Claude |
| 2 | Codebase index / repo-map (local embeddings) | Remember | rag-house.js (+ index lib) |
✅ |
| 3 | Search-replace edit + re-verify loop | Act/Verify | autowork + keystone-escalation.js |
✅ |
| 4 | 5-tier context compaction | Remember | assembleSessionContext() |
✅ |
| 5 | Isolated sub-agents | Reason | convergence-agent.js |
✅ |
Already at parity (keep): verify-gated escalation (KEYSTONE_LOCAL_FIRST), permission tiers (command-allowlist.js + safe-exec.js), worktree isolation, PCSF routing, CSF memory, convergence records, web grounding.
The whole harness is GPU-independent and verifiable today against cloud Claude — the local frontier model just slots into the same loop when the GPU lands.
5. Roadmap
| Phase | Scope | Needs GPU? | Outcome |
|---|---|---|---|
| 0a | VRAM auto-detect + register Qwen 3.6-27B (frontier capabilityScore); demote Ouro to research entry |
scaffolds now, inert untilGB box | Capability gate ready |
| 0b | Confirm exact Ollama/vLLM model tag; serve Qwen 3.6-27B; dogfood | yes (buy/rent) | Local frontier live |
| 1 | Uniform multi-turn agent loop + search-replace edits | no | The visible "feels like Claude Code" leap |
| 2 | Local codebase index / repo-map | no | Context quality + ownership story |
| 3 | 5-tier compaction | no | Long sessions + token margin |
| 4 | Isolated sub-agents | no | Devin-class capstone |
Recommended order: Phasenow (highest leverage, fully testable against Claude today, no waiting on hardware) + Phase 0a scaffolding in parallel (safe/inert onGB). Then 0b the moment the GPU arrives, then 2→4.
6. Open items for Alex
- Cloud cost model: product fronts it (subscription, cf. Patreon tiers) or BYO-key? Sets how hard Tierworks.
- GPU: buy (used RTX~$700–900 // 5090) or rent (RunPod/Vast/Lambda) — either works; the registry auto-detects.
- Build scope: confirm Phasenow + 0a scaffold (recommended), or different order?