Status: READY FOR INTEGRATION • Owner: Helix Ops • Last Updated: 2025‑10‑13
granite4-small-hibm/granite4:small-h (aka Granite 4.0 H‑Small)Sources: IBM Granite 4.0 model cards/repos + Ollama registry entries. Record the exact artifact you deploy.
Record on deploy:
source_url: (HF/GitHub/Ollama)artifact_sha256: (computed locally)artifact_size_bytes: (from filesystem)pull_command: (ollama pull ibm/granite4:small-h or HF path)signed_by: (if signed; attach attestation if present)helix_proof_id: (SHA‑256 of this HMI file)Hashing procedure:
# After pull/export to file (example path):
sha256sum /opt/models/ibm/granite4-small-h.gguf | tee /opt/helix/proofs/models/granite4-small-h.sha256 |
| Runtime |
|---|
| Status |
| Notes |
| Ollama |
| ✅ Primary |
| Official ibm/granite4:small-h images; multiple quantizations |
| vLLM |
| ✅ |
| Use HF checkpoint; ensure Mamba‑2 support flags enabled |
| Text‑Gen Inference (TGI) |
| ✅ |
| Load via transformers w/ Mamba‑2 kernels |
| LM Studio |
| ✅ |
| H‑Small listed; for local eval |
`# /opt/helix/registry/models/granite4-small-h.yaml model_id: granite4-small-h family: granite version: \"4.0\" upstream: ibm/granite4:small-h license: Apache-2.0 architecture: type: hybrid mix: [transformer, mamba2] moe: total_parameters_b: 32 active_parameters_b: 9 context: recommended_temperature: 0 max_new_tokens_default: 512 quantizations:
name: Q4_K_M approx_size_gb: 19
name: Q5_K_M approx_size_gb: 23 capabilities: instruction_following: strong tool_use: strong code: medium languages: [en, de, es, fr, ja, pt, ar, cs, it, ko, nl, zh] qsr_profile: granite4-small-h-2025Q4 policy: pii_scrub: helix-default safety_tier: standard audit_headers: true vector_memory: collection: ttd_memory_v2 ethos_tag: ethos_core_v1`
Use this single source of truth system prompt for Granite4 in Helix. It requires your router/middleware to fetch the Helix Core Ethos from Qdrant and inject it into the placeholder before sending to the model.
Router responsibility (pseudocode):
ethos = qdrant.search(collection=\"ttd_memory_v2\", query=\"Helix Core Ethos\", filter={\"must\": [{\"key\":\"tags\",\"match\": {\"any\": [\"ethos\",\"ethos_core_v1\"]}}]}, top_k=1)
ETHOS_SNIPPET = ethos[0].payload.get(\"output\") or ethos[0].payload.get(\"text\")#Fallback to cached file
if not ETHOS_SNIPPET: ETHOS_SNIPPET = open(\"/opt/helix/cache/ethos_core_v1.txt\").read()
System Prompt Template (inject ETHOS_SNIPPET):
`You are a Helix assistant operating under the Helix Core Ethos. Use the following canonical ethos excerpt as your ground truth:
[HELIX_CORE_ETHOS]
{{ETHOS_SNIPPET}}
[/HELIX_CORE_ETHOS]
POLICY:
OPERATING MODES:
Ollama example (direct):curl -fsS http://127.0.0.1:11434/api/generate -d '{
\"model\": \"ibm/granite4:small-h\",
\"system\": \"Helix router (optional) message shape:{
\"route\": \"granite4-small-h\",
\"messages\": [
{\"role\":\"system\",\"content\":\"Guardrails: If ETHOS_SNIPPET cannot be loaded, abort the request withHTTP 503andreason=ethos_unavailable` (do not proceed without ethos context).Store provenance with each completion in VectorDB (Qdrant) and mirror to filesystem proofs. Use this canonical envelope as payload.audit when upserting a point.
{
\"model\": \"granite4-small-h\",
\"vendor\": \"ibm\",
\"version\": \"4.0\",
\"artifact_sha256\": \"<fill-from-provenance>\",
\"quantization\": \"Q4_K_M|Q5_K_M|bf16\",
\"inference_stack\": \"ollama|vllm|tgi\",
\"router\": \"helix-router|direct-ollama\",
\"source\": {
\"pulled_at\": \"<iso8601>\",
\"channel\": \"ollama|hf|github\",
\"attestation\": \"<optional>\"
},
\"session\": {
\"id\": \"<uuid>\",
\"user\": \"helix\",
\"ts\": \"<iso8601>\"
}
}
Filesystem mirror (recommended): write the same envelope to /opt/helix/proofs/sessions/<session-id>/audit.json.
Use these as initial thresholds; tune with live telemetry over the first week. `# /opt/helix/qsr/profiles/granite4-small-h.yaml profile_id: granite4-small-h-2025Q4 weights: coherence: 0.28 accuracy: 0.26 completeness: 0.18 relevance: 0.18 novelty: 0.10 thresholds: soft_flag: 0.74 # trigger human‑in‑the‑loop note hard_block: 0.62 # route to rollback / alternative model mri_risk_tiers: low: [0.80, 1.00] medium: [0.70, 0.80) high: [0.00, 0.70) fallback_chain:
model: deepseek-coder` Calibration plan:
Run 200‑sample eval across Helix golden tasks (RAG, tool‑use, code review).
soft_flag to equalize FP/FN at δ≤2%.A) Pull & list (Ollama):
ollama pull ibm/granite4:small-h
ollama list | grep granite4
B) Direct inference (no TTD shim):
# Simple chat via Ollama
curl -fsShttp://127.0.0.1:11434/api/generate-d '{
\"model\": \"ibm/granite4:small-h\",
\"prompt\": \"Summarize the Helix Core Ethos in 3 bullets.\",
\"stream\": false
}' | jq
C) (Optional) Helix router entry: if you still route via Helix router, keep the minimal model map but no /chat shim dependency.
{
\"route\": \"granite4-small-h\",
\"backend\": \"ollama\",
\"model\": \"ibm/granite4:small-h\",
\"parameters\": {\"temperature\": 0, \"num_ctx\": 32768}
}
Restart router only if used:
sudo systemctl restart helix-router || true
top_p=0.9, max_new_tokens=512; penalize repetition >1.1tool_timeout_ms=8000ttd_memory_v2JSON schema example:
{
\"name\": \"lookup_kb\",
\"strict\": true,
\"parameters\": {
\"type\": \"object\",
\"properties\": {
\"query\": {\"type\": \"string\"},
\"top_k\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 10}
},
\"required\": [\"query\"],
\"additionalProperties\": false
}
}
Two paths: A) full VectorDB upsert with embeddings, B) proofs‑only if embeddings not available yet.
1. Embed the prompt+output (Ollama embeddings):
PROMPT='Good morning Helix and welcome Granite4:small-h, please review the Helix Core Ethos.'
# Replace with your embeddings model route if different
EMB_MODEL=\"nomic-embed-text\"
VEC=$(curl -fsShttp://127.0.0.1:11434/api/embeddings-d \"{\\"model\\":\\"$EMB_MODEL\\",\\"prompt\\":\\"$PROMPT\\"}\" | jq -c '.embedding')
2. Generate with Granite 4:
OUT=$(curl -fsShttp://127.0.0.1:11434/api/generate-d \"{\\"model\\":\\"ibm/granite4:small-h\\",\\"prompt\\":\\"$PROMPT\\",\\"stream\\":false}\" | jq -r '.response')
3. Build audit envelope and upsert into Qdrant:
`SESSION=$(uuidgen)
SHA=$(cat /opt/helix/proofs/models/granite4-small-h.sha256 2>/dev/null | awk '{print $1}')
cat > /tmp/audit.json <<JSON
{
\"model\": \"granite4-small-h\",
\"vendor\": \"ibm\",
\"version\": \"4.0\",
\"artifact_sha256\": \"${SHA:-unknown}\",
\"quantization\": \"Q4_K_M|Q5_K_M|bf16\",
\"inference_stack\": \"ollama\",
\"session\": {\"id\":\"$SESSION\",\"user\":\"helix\",\"ts\":\"$(date -Iseconds)\"}
}
JSON
# Upsert (default Qdrant localhost:6333; collection ttd_memory_v2)
cat > /tmp/qdrant_upsert.json <<JSON
{
\"points\": [
{
\"id\": \"$SESSION\",
\"vector\": $VEC,
\"payload\": {
\"type\": \"completion\",
\"prompt\": \"$PROMPT\",
\"output\": \"$OUT\",
\"audit\": $(cat /tmp/audit.json)
}
}
]
}
JSON
curl -fsS -X POST 'http://127.0.0.1:6333/collections/ttd_memory_v2/points' \
-H 'content-type: application/json' \
-d @/tmp/qdrant_upsert.json | jqExpected:\"status\":\"ok\"and\"operation_id\"` in response.
SESSION=$(uuidgen)
mkdir -p /opt/helix/proofs/sessions/$SESSION
printf '%s' \"$OUT\" > /opt/helix/proofs/sessions/$SESSION/output.txt
cp /tmp/audit.json /opt/helix/proofs/sessions/$SESSION/audit.json
sha256sum /opt/helix/proofs/sessions/$SESSION/output.txt | tee /opt/helix/proofs/sessions/$SESSION/output.sha256
helix_granite4_small_h_* (latency, tokens, qsr, blocks, fallbacks).magnus-supernova + qwen3-7b; automatic switchover on hard_block.A) Incantations `# Quantized alt pulls ollama pull ibm/granite4:small-h-q4_K_M ollama pull ibm/granite4:small-h-q5_K_M
# vLLM (HF)
python - <<'PY'
from vllm import LLM
llm = LLM(model=\"ibm-granite/granite-4.0-h-small\", dtype=\"bfloat16\")
print(llm.generate([\"Hello Granite 4!\"]))
PYB) Consent stub (TTD){
\"consent_id\": \"ttd-consent-granite4-small-h-2025-10-13\",
\"subject\": \"Deployment of IBM Granite 4.0 H-Small in Helix\",
\"artifacts\": [
{\"name\": \"granite4-small-h.gguf\", \"sha256\": \"