//Distributed AICompute

Components

The Platform Has Four Core Layers

A worker agent on every machine, a control plane that coordinates them, a runtime that executes inference, and an API that applications consume. A dashboard sits on top for visibility.

Component 01

Node Agent

The worker software running on every participating computer.

The Node Agent is a lightweight Go-based service installed on each compute machine. It owns the machine’s identity in the cluster, reports what hardware it has, and executes the work the controller assigns to it.

Responsibilities

  • Persistent node identity
  • Node registration
  • Authentication / enrollment
  • Hardware detection
  • GPU / CPU / RAM / VRAM reporting
  • Heartbeats
  • Telemetry
  • Job lifecycle management
  • Runtime integration
  • Inference execution
  • Cancellation
  • Drain mode
  • Reconnection
  • Graceful shutdown
  • Structured logging
  • Health reporting
NODE AGENT LIFECYCLE

Install
  ↓
Generate / Load Node Identity
  ↓
Connect to Controller
  ↓
Authenticate
  ↓
Register
  ↓
Report Hardware
  ↓
Heartbeat
  ↓
Receive Work
  ↓
Execute
  ↓
Stream Results
  ↓
Continue Heartbeats
ONLINE
READY
BUSY
DRAINING
OFFLINE
UNHEALTHY

Node Agent

Deploy a New Compute Node

Install the Node Agent on another computer and add it to the cluster.

Windows

node-agent-windows.exe

Download for Windows

Linux

node-agent-linux

Download for Linux

Source

Build from source

View Source Code
powershellinstallation example
.\install.ps1 `
  -ControllerURL "YOUR_CONTROLLER_URL" `
  -EnrollmentToken "YOUR_ENROLLMENT_TOKEN"

Platform buttons download the latest uploaded build for that platform. The enrollment token should be generated securely by the cluster administrator and never committed or published.

Component 02

Cluster Controller

The control plane that coordinates the distributed compute cluster.

The Cluster Controller is the centralized control plane, deployed independently from the worker machines. It knows which nodes exist, what state they are in, and which of them should participate in a given workload.

Responsibilities

  • Node registration
  • Authentication
  • Node state management
  • Health monitoring
  • Heartbeat processing
  • Telemetry collection
  • Resource tracking
  • Scheduling
  • Job management
  • Distributed orchestration
  • Model management
  • Admin APIs
  • Event streaming
  • Node draining
  • Node recovery coordination
                    CLUSTER CONTROLLER
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
        ▼                  ▼                  ▼
   Node Registry       Scheduler          Job Manager
        │                  │                  │
        └──────────────────┼──────────────────┘
                           │
                    Health Monitor
                           │
                           ▼
                    Event / Admin API

The controller is currently deployed on a cloud platform and acts as the control plane only. The cloud platform is not the inference engine.

Component 03

Distributed Inference Runtime

The execution layer responsible for actually running distributed AI inference.

The Controller does not perform the actual LLM inference. It coordinates the compute nodes. The distributed runtime performs the inference work across the participating machines.

Controller
    │
    ▼
Workload / Execution Plan
    │
    ▼
Distributed Runtime
    │
 ┌──┼──────────┐
 ▼  ▼          ▼
N1  N2         N3
 │   │          │
 └───┼──────────┘
     ▼
  LLM Runtime
     │
     ▼
Generated Tokens

Current runtime stack

  • Distributed runtime
  • llama.cpp
  • GGUF model
  • GPU / CPU compute

Only the runtime pieces listed above are integrated. Support for every model format or every inference runtime is not claimed.

Component 04

AI API Layer

How applications consume the cluster.

The platform exposes an AI API that applications can consume without directly communicating with individual compute nodes. Applications interact through a familiar API interface while the infrastructure handles compute-node coordination behind the scenes.

Application
     │
     ▼
OpenAI-Compatible API
     │
     ▼
Cluster Controller
     │
     ▼
Distributed Compute
envplaceholders only
AI_BASE_URL=https://YOUR-CONTROLLER/v1
AI_API_KEY=YOUR_API_KEY
AI_MODEL=YOUR_MODEL

Project Scout is one application that can consume this API.

Component 05

Cluster Dashboard

Centralized visibility into the cluster.

  • Cluster health
  • Nodes
  • Node hardware
  • Jobs
  • Models
  • Events
  • Node status
  • Resource information
  • Drain / resume operations
              Web Dashboard
                    │
                    ▼
             Admin REST API
                    │
                    ▼
           Cluster Controller

The dashboard never communicates directly with worker nodes.

Build Your Own AI Compute Cluster

Connect your machines. Deploy the Node Agent. Start the controller. Build a distributed AI environment around the hardware you already have.

Download Node AgentExplore ArchitectureView DocumentationGitHub