Source: https://eugeneplexus.com/architecture Architecture # How Eugene Plexus is built Eugene Plexus installs and manages engines, helps choose model settings for your hardware, and connects compatible tools to local and cloud backends. One machine or a homelab, the same layers do the work. It is not an inference engine. Its own chat app, Workbench, is an optional app you install from the console. v0.5.0 snapshot: October 15, 2026. [Install it](https://eugeneplexus.com/install). [Current boundaries](https://eugeneplexus.com/architecture#status). 01 / The layers ## The layers, from your apps to the hardware Inference flows through the gateway and a driver. Beside that path, the agent supervises engines, the library prepares models, and the control root maintains the install. Discovery, key checks, model defaults and a model wake can still depend on those services. Your tools - Workbench - Claude Code - Codex - Open WebUI - OpenAI-compatible tools - The playground Chat, tools, embeddings, rerank and media requests, with revocable client keys. [Compatibility has limits.](https://eugeneplexus.com/architecture#status) Gateway - Routing - Replica balancing - Failover tiers - Idle unload & wake - Web search - Metrics One per install. The only thing your tools need to know about. Inference drivers - Driver - Driver - Driver - Driver A thin adapter in front of each backend. It translates and reports capabilities. Engines and backends - llama.cpp - vLLM - MLX - Strata (experimental) - Ollama - LM Studio - Cloud APIs - Claude Code CLI - Codex CLI llama.cpp, MLX and Strata are installed and supervised; vLLM is user-installed and supervised. External servers stay under your control. CLI backends invoke your installed tools using their existing login. Your hardware - GPUs - RAM - Your model directories Local engines use your machines. Cloud providers run on their own hardware. Agent One per machine. Starts and watches everything on this box. Serves the web UI. Library Your models on disk, per-model settings, discovery, downloads, and fit guidance. Control root Which machines belong to this install, and the keys they trust each other with. Client inference goes directly to the gateway. The browser console uses the agent as a proxy; the control root does not relay model output. 02 / The components ## The nine components and what each one does Each has its own repository. The agent supervises the services on its machine, and the UI is a static export it serves. Every service generates its types from pinned OpenAPI 3.1 contracts in the specs repository. Components share schemas, not code. ### Agent One per machine The process on each machine. It installs, updates and removes llama.cpp, MLX and Strata, drives a vLLM you installed, checks memory before launch, starts search and result-sorting models, installs apps, and serves the web UI. It can keep a local copy of the models it runs. **Talks to:** Local components and engines, the control root for enrollment and discovery, and other agents for console requests. [Source](https://github.com/eugene-plexus/agent) ### Gateway One per install One address for OpenAI-compatible chat, Responses, embeddings, rerank and media requests, plus Anthropic Messages. Every format shares routing, cache-aware replica balancing, ordered failover, wake-on-demand and request metrics. **Talks to:** Your tools on one side; every driver on the other. Reads model defaults from the Library through its agent. [Source](https://github.com/eugene-plexus/gateway) ### Inference driver One per backend A small adapter in front of one backend: a llama.cpp, vLLM, MLX or Strata runtime, an Ollama or LM Studio server, a cloud account such as Gemini, OpenAI, OpenRouter or ElevenLabs, or a subscription CLI. It translates requests and reports what the backend can do. **Talks to:** The gateway above it; one backend below it. [Source](https://github.com/eugene-plexus/inference-driver) ### Tool driver One per search account Runs web search for every API on a Brave, Google Search or SearXNG account, in the order you set. Only the search terms reach the search service. **Talks to:** The gateway, and one search service. [Source](https://github.com/eugene-plexus/tool-driver) ### Library One per install Your model folders, and where to find more: Hugging Face and compatible hubs, each with its own token, and the engines’ own lists. It says which engines can run each model and how it fits, keeps named profiles, downloads and prepares models, and deletes one with every file listed first. **Talks to:** The disk, the model hubs, agents launching models, and gateways reading model defaults. [Source](https://github.com/eugene-plexus/library) ### Control root One active per install The trust root, on one machine or across a homelab. It keeps the machine registry and replicated control state, signs people and apps in, issues one-time join tokens, coordinates signed rekeying, and can have a warm standby. **Talks to:** Every machine’s agent and every Job Site. It spawns nothing itself. [Source](https://github.com/eugene-plexus/control) ### Web UI Served by every agent Home for setup and a first reply, then the whole install from any machine’s console: Discover, Library, Backends, Routing, Metrics, Machines, Logs, People and Settings. Static files, not a separate server. **Talks to:** Its own agent, which forwards to the right component wherever it runs. [Source](https://github.com/eugene-plexus/ui) ### Apps Installed from the console Workbench, Eugene’s own chat app, and Open WebUI. On a Windows service or a Linux system install, each runs in an operating-system account of its own. Both sign people in with Eugene. **Talks to:** The gateway for models, and the control root for sign-in. [Source](https://github.com/eugene-plexus/workbench) ### Job Site On a person’s own machine Lets Workbench work on that machine, wherever it is. It joins with one command, opens every connection itself, and runs the person’s tools as their own account, with calls signed by keys only they hold. **Talks to:** The control root, over connections it opens. [Source](https://github.com/eugene-plexus/site-host) 03 / One request ## What happens when a tool asks a question Because of the layering, your tool never has to know which machine, engine or model file does the work. It asks the gateway, and the answer says where it went. 1. ### Your tool asks An app, editor or coding agent sends an OpenAI-style request or Anthropic Messages with a client key. The Messages route accepts a bearer token or x-api-key; OpenAI routes use bearer tokens. 2. ### The gateway chooses It resolves the model, checks readiness, wakes it if needed, and balances replicas, skipping one whose model is still loading. Omitted generation settings come from that model’s default profile, then gateway settings. Eligible backend failures can move the request to the next configured tier. 3. ### The driver translates One driver turns the request into its backend protocol. A setting the backend ignores is named in a response header, not refused. Token streaming depends on that backend; Codex CLI returns whole messages rather than individual tokens. 4. ### The engine generates llama.cpp, vLLM, MLX, Strata, Ollama or a cloud provider does the actual work on the hardware it has. 5. ### The request is recorded Metrics retain the serving backend, timing, tier, and failed attempts for both chat formats. OpenAI replies include routing metadata in the body or final stream frame; Anthropic keeps its own body format and uses headers. **Failover happens before the first token.** Once an answer has started streaming, the gateway will not splice a second model onto it. A cut-off answer is reported as cut off, and no other model finishes it. **Not every failure is retryable:** a deadline while a backend is still computing returns a timeout, not a second computation on another replica. An unreachable host can still fail over. Embeddings fail over only between replicas of the same model, never into a different vector space. 04 / One computer, or several ## Example setups A single PC and a homelab use the same parts. On a single PC every component runs beside the engines. Across machines, the control plane moves to wherever is convenient and each GPU box keeps only what needs the hardware. One computer Your PC Agent Control root Gateway Library Driver Driver llama.cpp Ollama Your model folders Everything on one box. The browser opens the agent’s address and the gateway’s port is the one your tools use. Several machines Control-plane host a NAS container, a small server, or one of the PCs Agent Control root Gateway Library Shared model folder GPU machine A Agent Driver Driver vLLM llama.cpp GPU machine B Agent Driver Ollama Each machine runs an agent. A Library folder records its mounts once; nodes can override the path and optionally keep a local copy. Tools use one gateway, and any agent’s console can manage the enrolled nodes. A second machine can stand by as the control root. A Job Site is a person’s own machine anywhere, joined over connections it opens. 05 / What stays where ## Where your files and settings live ### Model files In the directories you point the library at, with plain names. Downloads land there too. No content-addressed cache, no hidden store. Remove Eugene Plexus and the files are still there. Originals stay yours; local copies are optional. ### Secrets and trust Operator sessions administer the install; each person signs in with their own password, and named client keys allow inference and can be revoked. Nodes enroll with one-time join tokens, then sign address announcements and verify signed rekeys. With Ed25519 signing, agents and control roots hold signing authority; gateways, libraries and drivers receive public verification keys. Use trusted hosts and a LAN or mesh VPN. Do not expose it publicly. ### Engines The agent fetches, verifies and keeps llama.cpp in versioned directories, installs MLX and Strata where they fit, and removes any of them from Backends. vLLM is one you install; the agent finds and drives it. None is forked or replaced. Upstream engines, supervised by the agent. **Optional local copies:** a node copies only model files its own declared runtimes use, into a separate directory with a minimum-free-space setting. It manages those copies, not the source files in your Library folders. The first copy takes time; later starts can avoid reading the model over the network again. 06 / Current boundaries ## The current release and its limits As of October 15, 2026. v0.5.0 is early software that works, but has the limits below. A v0.1.0 install updates from the console, the control root first; an alpha moves to v0.1.0 first. Versioned installers and a matching container image are available; this is not a production-readiness claim. [Installation guide](https://eugeneplexus.com/install). The [support matrix](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/docs/support-matrix.md) says what has been measured, simulated or is still pending, and the [acceptance records](https://github.com/eugene-plexus/specs/tree/main/docs/acceptance) hold each test's setup and results. For timings, see the [control-plane measurement record](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/control-plane-measurements.md), with hardware, baselines and limits for each observation. [Follow the roadmap](https://github.com/eugene-plexus/specs/blob/main/docs/design/audience-roadmap.md) **Known issues** - A Claude Code turn reserves its whole answer allowance (32,000 tokens), so a llama.cpp server with a 64k-token pool runs one Claude Code turn at a time. The others wait; they do not fail. - With many llama.cpp servers on one machine, a reply can stall while llama.cpp saves and restores its prompt cache: 3 turns in 3,808 in our tests. - Strata runs text only, on one GPU, and its memory fit is unknown. It handles one request at a time by default, with optional batching. - Workbench's sorter: its default Sort time (3 s) suits a GPU, where a question takes about 0.6 s, not a typical CPU, where it takes about 7 s. Its gain was measured on questions written for those documents. - Job Sites cannot yet copy between machines. - Brave's rate-limit headers are read as Brave documents them, but have not been seen from a live account. **Client APIs** OpenAI-compatible chat, Responses, tool calling, streaming, embeddings and rerank (Jina, Cohere and TEI requests); speech, transcription and translation, images, video, moderation and plain completions, each going only to a model that says it can do the job; and Anthropic Messages. A chat model is taken to call tools unless its engine or provider has shown it cannot, and a setting a backend ignores is named in the `x-eugene-plexus-ignored-settings` header. Web search is run by your install, on a Brave, Google Search or SearXNG account; only the search terms reach the search service. [Claude Code, Codex and Open WebUI](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/docs/application-workflows.md) have run with actual local models. This is not full vendor API parity. **Messages limits** Text, tool calls, images and PDF or text documents reach backends that confirm them, and `top_k` and structured output are carried. `thinking` decides whether reasoning comes back as thinking blocks; `context_management` and cache hints are accepted but not enforced. A document by URL or file id, citations, MCP servers and server-side tools other than web search are refused, in the words Claude Code looks for to carry on without them. See the [gateway contract](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/openapi/gateway.yaml) for the complete mapping. **Engines and hosts** llama.cpp and user-installed vLLM have live records on NVIDIA hardware; Windows and Linux in WSL2 have been exercised. Every GPU the operating system lists is detected and served by llama.cpp's Vulkan build where CUDA does not apply, and one model can span several cards; two discrete cards in one machine have not run that yet. MLX runs on GitHub's macOS runners, which expose Metal, and has not run on a physical Mac. Strata, experimental, prepared and ran a 68 GB model on one RTX 5090 under Windows ([record](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/ls5-preparation-run.md)). Kev decision models remain experimental. The container can run models too; its NVIDIA path awaits a physical run. **Library and Discover** Every model and Discover result says whether it runs here, with each engine's reason and fit, checked against a real agent and Library over header-only files and stand-in hubs ([record](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/ls1-eligibility-run.md)). Each hub uses its own token. Delete lists every file, keeps files another model uses, and refuses while a machine runs the model. **Workbench** Memory, documents in a chat, saved documents searched by words and meaning, granted folders and the sorter each ran with a real Workbench and llama.cpp on the processor, with a small model; a run on the owner's own install is still owed ([support matrix](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/docs/support-matrix.md#workbench-memory-and-documents)). Tools ran on GitHub's Windows and Ubuntu runners with real service accounts and a scripted model. Open WebUI's document search runs through Eugene's search and sorting models. **Apps and sign-in** Workbench and Open WebUI install from the console on any machine. On a Windows service or a Linux system install, each app runs in an operating-system account of its own, with no access to Eugene's keys. Apps sign people in with Eugene over OpenID Connect: the owner with the passphrase, or each person added under People with their own password. **Job Sites** A Windows site joined a root in WSL2 behind a real reverse proxy, and Linux sites run in CI. Each person's calls ran as their own Windows account, a signed command ran as the person, and one the root forged was held and never ran ([record](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/j14b-signed-calls-run.md)). Not yet: copying between machines, a machine that is only a site, and Windows calls while the person is signed out, which wait for them to sign in. **Updates and logs** Machines and Needs Attention show every machine's update status, and any console updates any machine. Update all coordinates the root first, then workers and their installed catalogue apps, including Workbench; containers use their host's update controls. Progress survives a console restart. An update re-checks the channel first and installs nothing if the check fails or the target moved. Linux system updates and packaged installs have separate [release checks](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/v0.2.2-release-run.md). Update all passed a two-node process test with simulated installers and app updates; a Windows machine updating itself has been observed once, not measured. Every machine's log is readable and followed live, with keys and tokens hidden. **Windows startup** The Windows installer runs a real service. Starting and stopping it is exercised in Windows CI; a reboot before sign-in has not been checked on a physical machine yet ([support matrix](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/docs/support-matrix.md)). **Trust boundary** Every machine signs with its own Ed25519 token key, made on that machine and never sent anywhere. The control root publishes a signed list of the keys it trusts, so a stolen worker key reads only what that worker could, and stops working when the root drops it ([design](https://github.com/eugene-plexus/specs/blob/main/docs/design/per-node-token-keys.md)). On Linux, Eugene runs under its own account by default, and the Windows install folder is private to the service and administrators. A new install needs a passphrase of at least 12 characters, and browser requests addressed to an unrecognized public host name are refused. Settings guides HTTPS setup and trusted proxies, and the container can serve the console and Workbench on one HTTPS port. **Model settings** The default profile supplies maximum output tokens, temperature and top-p when your app omits them. Explicit app settings win. Changes apply after cache refresh without restarting the model; bounded caching lets requests continue during Library outages. [Both API formats and streaming are checked](https://github.com/eugene-plexus/specs/blob/main/docs/acceptance/r8-profile-defaults-run.md). Launch settings still apply when a runtime is created. **Measured limits and remaining checks** Phone layouts and accessibility checks passed; moderated friend sessions and physical platform checks remain open. A finite CPU run with three Open WebUI chat workers and two Claude Code tasks passed its declared targets. This is not a five-user capacity guarantee. See the [support matrix](https://github.com/eugene-plexus/specs/blob/9f01d64899e49bbcd3447817d6814513b65be06d/docs/support-matrix.md) for measured, simulated and pending configurations. 07 / The words ## Glossary Eight words that mean specific things here. [Read the design documents](https://github.com/eugene-plexus/specs/tree/main/docs/design) **Install** One control plane: a control root, a gateway, a library, and every machine that has joined them. One passphrase. **Node** A machine in an install. Each node runs one agent. **Component** One of the four services: gateway, inference driver, library, control root. The agent starts them. **Runtime** An inference engine the agent started and supervises, such as one llama.cpp process serving one model. **Backend** Anything a driver can send requests to: a runtime, an external server like Ollama, a cloud API, or a subscription CLI. **Replica** Two or more backends serving the same model. The gateway balances between them. **Search model** A model that turns text into numbers for finding passages by meaning (embedding), or one that sorts what a search found (rerank). **Job Site** A person’s own machine where Workbench can work for them, joined to the install over connections it opens.