← Back to Whitepapers

ForgeChainOS Desktop Phase 1: Inference Wiring Spec

Author: NOAH (963, Ark conductor)
Date: 2026-08-18
Status: SPECIFICATION. Family-internal. SCAR #5 compliant.
Authority: Node Zero. NODEZEROINSIDE.

1. Disable Jan's Built-In Inference (HOTCLUTCH Owns GPU)

1.1 The Problem

Jan ships with two inference extensions:
- llamacpp-extension: Spawns a local llama.cpp router process, downloads models from HuggingFace, manages GPU memory
- mlx-extension: MLX-Swift inference on Apple Silicon (macOS-only, already excluded on Linux by build-time IS_MACOS flag)

If llamacpp-extension loads, it will:
1. Attempt to spawn its own llama.cpp process (fighting hermes-brain for port :8093 or GPU memory)
2. Download models from HuggingFace (bandwidth waste, disk waste)
3. Manage GPU lifecycle (HOTCLUTCH owns this on BH)

1.2 The Fix: Comment Out llamacpp-extension in bundled-extensions.ts

File: /home/nodezero/quarantine/jan/web-app/src/services/core/bundled-extensions.ts

Lines 51-56 contain the llamacpp-extension entry in the ENTRIES array:

  {
    load: () => import('@janhq/llamacpp-extension'),
    name: '@janhq/llamacpp-extension',
    productName: 'llama.cpp Inference Engine',
    version: '1.0.1',
    description: 'This extension enables llama.cpp chat completion API calls',
  },

Action: Comment out this entry. This is a BUILD-TIME change. After commenting, rebuild Jan (yarn build). The resulting binary will not load llamacpp-extension at all.

  // FORGECHAINOS: llamacpp-extension DISABLED. HOTCLUTCH owns GPU.
  // hermes-brain :8093 is the sole inference provider via OpenAI-compatible remote.
  // {
  //   load: () => import('@janhq/llamacpp-extension'),
  //   name: '@janhq/llamacpp-extension',
  //   productName: 'llama.cpp Inference Engine',
  //   version: '1.0.1',
  //   description: 'This extension enables llama.cpp chat completion API calls',
  // },

MLX is already disabled on Linux. Line 77: if (IS_MACOS) gates the mlx-extension. BH is Linux. No action needed.

1.3 Impact of Disabling llamacpp

Feature	Without llamacpp-extension	Mitigation
Local model download	DISABLED	Not needed. We provide our own models via HOTCLUTCH.
Local llama.cpp process spawn	DISABLED	hermes-brain is the process, managed by systemd + HOTCLUTCH.
GPU memory management	DISABLED	HOTCLUTCH :7796 manages GPU lease state machine.
Model loading/unloading	DISABLED	HOTCLUTCH /api/swap handles model changes.
Chat completions	PRESERVED	Jan routes to remote providers (OpenAI-compatible). hermes-brain IS the remote provider.

1.4 Extensions That STAY

Extension	Purpose	Needed?
assistant-extension	Default AI assistant behavior	YES (manages conversation flow)
conversational-extension	Conversation persistence	YES (saves chat history)
download-extension	File download management	OPTIONAL (not needed if no model downloads, but may be used by other features)
rag-extension	RAG tool orchestration	OPTIONAL (we have our own ORNITH/FORGEVECTOR, but may complement)
vector-db-extension	Local vector DB	OPTIONAL (we have FORGEVECTOR :7803)

Keep all except llamacpp for Phase 1. In Phase 2, our custom ForgeChainOS extensions may replace rag-extension and vector-db-extension.

2. Wire hermes-brain as Sole Inference Provider

2.1 How Jan Remote Providers Work

Jan's model factory (model-factory.ts) supports remote providers via the createOpenAICompatible() function from @ai-sdk/openai-compatible. A remote provider needs:

Provider name (arbitrary string, e.g., "hermes-brain")
Base URL (the OpenAI-compatible API endpoint, e.g., http://127.0.0.1:8093/v1)
API key (can be empty/dummy for local endpoints)
Model list (auto-discovered from /v1/models endpoint)

2.2 Jan's Provider Configuration UI

Jan allows adding custom providers via Settings > Providers > "Add Provider":
- Name: hermes-brain
- Base URL: http://127.0.0.1:8093/v1
- API Key: forge-sovereign (dummy value, hermes-brain does not require auth)

This is a RUNTIME configuration stored in Jan's data directory.

2.3 Build-Time Provider Configuration (RECOMMENDED)

To bake hermes-brain as the default (and only) provider, add it to the predefinedProviders array in:

File: /home/nodezero/quarantine/jan/web-app/src/constants/providers.ts

Add at the TOP of the predefinedProviders array (line 54):

  {
    active: true,
    api_key: 'forge-sovereign',
    base_url: 'http://127.0.0.1:8093/v1',
    explore_models_url: '',
    provider: 'hermes-brain',
    settings: [
      {
        key: 'base-url',
        title: 'Hermes Brain URL',
        description:
          'BH sovereign inference endpoint. Managed by HOTCLUTCH. NODEZEROINSIDE.',
        controller_type: 'input',
        controller_props: {
          placeholder: 'http://127.0.0.1:8093/v1',
          value: 'http://127.0.0.1:8093/v1',
        },
      },
    ],
    models: [
      {
        id: 'hermes-brain',
        name: 'Hermes-3 Llama 3.1 8B (Q4_K_M)',
        capabilities: ['completion'],
      },
    ],
  },

2.4 Verification (hermes-brain is ready)

hermes-brain is ALIVE and serving OpenAI-compatible API (verified this session):

$ curl -sf http://localhost:8093/v1/models
{
  "models": [{"name": "hermes-brain", ...}],
  "data": [{"id": "hermes-brain", "aliases": ["hermes-brain"], ...}]
}

Test chat completion:

curl -sf http://localhost:8093/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"hermes-brain","messages":[{"role":"user","content":"Hello"}],"max_tokens":50}'

2.5 Verification That Jan Routes Through hermes-brain

After the build with these changes:
1. Launch Jan
2. Open Settings > Providers: hermes-brain should appear as active
3. Select hermes-brain as the model in the chat view
4. Send a message
5. Check hermes-brain logs: journalctl -u hermes-brain -f should show the inference request
6. Verify NO llamacpp process: pgrep -f llamacpp should return nothing from Jan (hermes-brain's own llama-server process will still be there)
7. Verify GPU ownership: nvidia-smi should show only hermes-brain's GPU allocation, not a separate Jan process

2.6 Configuration Hierarchy

HOTCLUTCH :7796 (GPU lease state machine)
  |
  +-- Manages hermes-brain (8B, CPU default, GPU on lease)
  +-- Manages ORNITH (9B, GPU when leased)
  +-- /api/swap: kill old model, start new, verify
  |
  v
hermes-brain :8093 (llama-server, OpenAI-compatible)
  |
  +-- /v1/models (model discovery)
  +-- /v1/chat/completions (inference)
  |
  v
ForgeChainOS Desktop (Jan, Tauri shell)
  |
  +-- HTTP client to hermes-brain (via OpenAI-compatible provider)
  +-- Display layer ONLY: sends HTTP requests, does NOT manage processes
  +-- No model download, no GPU control, no process spawning

Jan is a DISPLAY LAYER. It sends HTTP. HOTCLUTCH owns the GPU. hermes-brain owns the inference. This separation is load-bearing.

3. Build Instructions (Phase 1 Execution)

cd /home/nodezero/quarantine/jan

# 1. Disable llamacpp-extension (edit bundled-extensions.ts)
# Comment out the llamacpp entry as described in Section 1.2

# 2. Add hermes-brain provider (edit providers.ts)
# Add the hermes-brain entry as described in Section 2.3

# 3. Rebuild
yarn build

# 4. Test
./src-tauri/target/release/Jan
# OR: cargo tauri dev (for development mode with hot reload)

Build Prerequisites (already confirmed on BH)

Rust 1.95.0 + cargo (CONFIRMED)
Node.js v20.18.0 (CONFIRMED)
yarn (NEEDS: corepack enable && corepack prepare yarn@4.5.3 --activate)
libgtk-3-dev (CONFIRMED)
libwebkit2gtk-4.1-dev (CONFIRMED)
libappindicator3 (CONFIRMED)

4. Future: NOAH Router Integration (Phase 2)

Phase 1 wires hermes-brain directly (raw inference, no corpus grounding).
Phase 2 (the NOAH Chat extension) adds:
- Corpus grounding via NOAH router :7743
- ON-PARR truth gate verification
- ORNITH RAG :8095 for context injection
- Tier-gated access (IMPERIAL/beta/external)

The hermes-brain wiring from Phase 1 remains as the fallback direct inference path.

NODEZEROINSIDE.