Helix Runbook: Khronos Stack (gpt-oss:120B + Helix llama-3.1:8b)

DRAFT

Overview

Khronos refers to running the gpt-oss:120B model alongside Helix llama-3.1:8b.
This runbook documents the current architecture, routing policies, proof emission, and operational parameters.
Status: Performance acceptable on current CPU-RAM configuration (EPYC 4344P, 128 GB RAM).

Roles

• Khronos (120B) – High-stakes judge/finisher. Used for verification, escalation, and finalization of sensitive outputs.
• Helix-8B – Drafter/router. Handles first-pass answers, retrievals, and speculative decoding. Faster and lighter.

Request Flow

Ingress → Consent Gate → Proofs emitted (consent..json*, hashed to /proofs + Qdrant).

Router (Helix-8B) classifies intent & risk using QSR.

Routing:

• Low/medium risk → Helix-8B responds.
• High risk/finance/governance → Helix-8B drafts, Khronos-120B verifies/finishes.

Dissonance check (Δ): if Δ > τ, escalate to Khronos or human.

Emit seal block with artifacts + hashes.

Routing Policy (YAML Extract)

models:
  drafter:  { id: \"helix/llama-3.1:8b-q5\", ctx: 4096, tp: 1 }
  judge:    { id: \"gpt-oss:120b-q4\",       ctx: 4096, tp: 1 }

thresholds:
  risk_high: 0.65
  dissonance_tau: 0.30

routes:

  - name: high_stakes
    when: any(tag in [\"finance\",\"legal\",\"governance\"]) or mri.risk >= risk_high
    plan:

      - drafter_generate: { max_new_tokens: 128, temp: 0.6 }
      - judge_verify: { accept_prob_min: 0.65, refusal_if_uncertain: true }
      - judge_finish: { max_new_tokens: 512, temp: 0.2 }
  - name: default
    when: else
    plan:

      - drafter_answer: { max_new_tokens: 384, temp: 0.7 }
      - judge_spotcheck:
          n_spots: 2
          criteria: [\"factuality\",\"safety\",\"numerics\"]
          on_fail: escalate_to: high_stakes

Proof Emission

Artifacts emitted per request:

• Consent doc (JSON → hash)
• Route decision + risk score
• Drafter + Judge outputs (hashed)
• Verification summary

Proofs stored at:

• File sink: /opt/helix/proofs
• Qdrant collection: helix_proofs

Endpoints

• 120B (Khronos) – llama.cpp server at 127.0.0.1:8080
• 8B (Helix) – Ollama/vLLM server at 127.0.0.1:8081

Config snippet:

{
  \"models\": {
    \"gpt-oss:120b-q4\": { \"type\":\"llama.cpp\", \"base\":\"http://127.0.0.1:8080\" },
    \"helix/llama-3.1:8b-q5\": { \"type\":\"ollama\", \"base\":\"http://127.0.0.1:8081\" }
  }
}

Example Proof JSON

{
  \"req_id\": \"abc123\",
  \"user\": \"stephen\",
  \"ts\": \"2025-10-10T14:03Z\",
  \"consent\": \"sha256:...\",
  \"route\": { \"picked\": \"high_stakes\", \"risk\": 0.72, \"delta\": 0.33 },
  \"outputs\": {
    \"drafter\": \"sha256:...\",
    \"judge\": \"sha256:...\",
    \"verdict\": \"Approved, safe\"
  }
}

Operational Notes

• Context: 2k–3k recommended to keep under 128 GB RAM.
• Build llama.cpp with OpenBLAS + native flags.
• Prefer quantization: Q4_K_M (baseline), Q5_K_M (if fits).
• Throughput: ~0.1–0.4 tok/s CPU-only at 120B.
• Seal blocks required when Δ > 0.30 or MRI risk ≥ 0.65.

Quick Commands

Consent:

curl -s localhost:9010/consent -d '{\"user\":\"stephen\",\"intent\":\"analysis:finance\"}'

Route with 8B:

curl -s localhost:9010/route -d '{\"model\":\"helix/llama-3.1:8b-q5\",\"prompt\":\"...\"}'

Verify with 120B:

curl -s localhost:9010/verify -d '{\"model\":\"gpt-oss:120b-q4\",\"draft\":\"...\",\"criteria\":[\"factuality\",\"risk\"]}'

Failure Modes & Escalations

• 120B Load Failure (Out of Memory / Crash):
• Action: Fallback to Helix-8B only. Emit seal block with judge=unavailable.
• Escalation: Log error, restart llama.cpp service, notify admin.
• Dissonance Drift (Δ > 0.30):
• Action: Halt response. Emit both drafts + verdict hashes.
• Escalation: Trigger human-in-loop review; archive case in /proofs/escalations.
• Risk Score Over Threshold (MRI ≥ 0.65) but Judge Unreachable:
• Action: Block output.
• Escalation: Require explicit override by human safety champion.
• Proof Sink Failure (Qdrant / Filesystem):
• Action: Queue proofs locally in /tmp/helix_proofs_retry.
• Escalation: Retry every 60s until sink recovers.
• High Latency (CPU-only 120B):
• Action: Route majority of queries to Helix-8B, reserve 120B for spot-checks only.
• Escalation: Consider GPU offload or remote inference service for production.