Architecture
Eugene Plexus installs and manages engines, helps choose model settings for your hardware, and connects compatible tools to local and cloud backends. One machine or a homelab, the same layers do the work. It is not an inference engine. Its own chat app, Workbench, is an optional app you install from the console.
v0.5.0 snapshot: . Install it. Current boundaries.
01 / The layers
Inference flows through the gateway and a driver. Beside that path, the agent supervises engines, the library prepares models, and the control root maintains the install. Discovery, key checks, model defaults and a model wake can still depend on those services.
Chat, tools, embeddings, rerank and media requests, with revocable client keys. Compatibility has limits.
One per install. The only thing your tools need to know about.
A thin adapter in front of each backend. It translates and reports capabilities.
llama.cpp, MLX and Strata are installed and supervised; vLLM is user-installed and supervised. External servers stay under your control. CLI backends invoke your installed tools using their existing login.
Local engines use your machines. Cloud providers run on their own hardware.
02 / The components
Each has its own repository. The agent supervises the services on its machine, and the UI is a static export it serves. Every service generates its types from pinned OpenAPI 3.1 contracts in the specs repository. Components share schemas, not code.
The process on each machine. It installs, updates and removes llama.cpp, MLX and Strata, drives a vLLM you installed, checks memory before launch, starts search and result-sorting models, installs apps, and serves the web UI. It can keep a local copy of the models it runs.
Talks to: Local components and engines, the control root for enrollment and discovery, and other agents for console requests.
SourceOne address for OpenAI-compatible chat, Responses, embeddings, rerank and media requests, plus Anthropic Messages. Every format shares routing, cache-aware replica balancing, ordered failover, wake-on-demand and request metrics.
Talks to: Your tools on one side; every driver on the other. Reads model defaults from the Library through its agent.
SourceA small adapter in front of one backend: a llama.cpp, vLLM, MLX or Strata runtime, an Ollama or LM Studio server, a cloud account such as Gemini, OpenAI, OpenRouter or ElevenLabs, or a subscription CLI. It translates requests and reports what the backend can do.
Talks to: The gateway above it; one backend below it.
SourceRuns web search for every API on a Brave, Google Search or SearXNG account, in the order you set. Only the search terms reach the search service.
Talks to: The gateway, and one search service.
SourceYour model folders, and where to find more: Hugging Face and compatible hubs, each with its own token, and the engines’ own lists. It says which engines can run each model and how it fits, keeps named profiles, downloads and prepares models, and deletes one with every file listed first.
Talks to: The disk, the model hubs, agents launching models, and gateways reading model defaults.
SourceThe trust root, on one machine or across a homelab. It keeps the machine registry and replicated control state, signs people and apps in, issues one-time join tokens, coordinates signed rekeying, and can have a warm standby.
Talks to: Every machine’s agent and every Job Site. It spawns nothing itself.
SourceHome for setup and a first reply, then the whole install from any machine’s console: Discover, Library, Backends, Routing, Metrics, Machines, Logs, People and Settings. Static files, not a separate server.
Talks to: Its own agent, which forwards to the right component wherever it runs.
SourceWorkbench, Eugene’s own chat app, and Open WebUI. On a Windows service or a Linux system install, each runs in an operating-system account of its own. Both sign people in with Eugene.
Talks to: The gateway for models, and the control root for sign-in.
SourceLets Workbench work on that machine, wherever it is. It joins with one command, opens every connection itself, and runs the person’s tools as their own account, with calls signed by keys only they hold.
Talks to: The control root, over connections it opens.
Source03 / One request
Because of the layering, your tool never has to know which machine, engine or model file does the work. It asks the gateway, and the answer says where it went.
An app, editor or coding agent sends an OpenAI-style request or Anthropic Messages with a client key. The Messages route accepts a bearer token or x-api-key; OpenAI routes use bearer tokens.
It resolves the model, checks readiness, wakes it if needed, and balances replicas, skipping one whose model is still loading. Omitted generation settings come from that model’s default profile, then gateway settings. Eligible backend failures can move the request to the next configured tier.
One driver turns the request into its backend protocol. A setting the backend ignores is named in a response header, not refused. Token streaming depends on that backend; Codex CLI returns whole messages rather than individual tokens.
llama.cpp, vLLM, MLX, Strata, Ollama or a cloud provider does the actual work on the hardware it has.
Metrics retain the serving backend, timing, tier, and failed attempts for both chat formats. OpenAI replies include routing metadata in the body or final stream frame; Anthropic keeps its own body format and uses headers.
Failover happens before the first token. Once an answer has started streaming, the gateway will not splice a second model onto it. A cut-off answer is reported as cut off, and no other model finishes it.
Not every failure is retryable: a deadline while a backend is still computing returns a timeout, not a second computation on another replica. An unreachable host can still fail over. Embeddings fail over only between replicas of the same model, never into a different vector space.
04 / One computer, or several
A single PC and a homelab use the same parts. On a single PC every component runs beside the engines. Across machines, the control plane moves to wherever is convenient and each GPU box keeps only what needs the hardware.
Everything on one box. The browser opens the agent’s address and the gateway’s port is the one your tools use.
Each machine runs an agent. A Library folder records its mounts once; nodes can override the path and optionally keep a local copy. Tools use one gateway, and any agent’s console can manage the enrolled nodes. A second machine can stand by as the control root. A Job Site is a person’s own machine anywhere, joined over connections it opens.
05 / What stays where
In the directories you point the library at, with plain names. Downloads land there too. No content-addressed cache, no hidden store. Remove Eugene Plexus and the files are still there.
Originals stay yours; local copies are optional.Operator sessions administer the install; each person signs in with their own password, and named client keys allow inference and can be revoked. Nodes enroll with one-time join tokens, then sign address announcements and verify signed rekeys. With Ed25519 signing, agents and control roots hold signing authority; gateways, libraries and drivers receive public verification keys.
Use trusted hosts and a LAN or mesh VPN. Do not expose it publicly.The agent fetches, verifies and keeps llama.cpp in versioned directories, installs MLX and Strata where they fit, and removes any of them from Backends. vLLM is one you install; the agent finds and drives it. None is forked or replaced.
Upstream engines, supervised by the agent.Optional local copies: a node copies only model files its own declared runtimes use, into a separate directory with a minimum-free-space setting. It manages those copies, not the source files in your Library folders. The first copy takes time; later starts can avoid reading the model over the network again.
06 / Current boundaries
As of October 15, 2026. v0.5.0 is early software that works, but has the limits below. A v0.1.0 install updates from the console, the control root first; an alpha moves to v0.1.0 first. Versioned installers and a matching container image are available; this is not a production-readiness claim. Installation guide.
The support matrix says what has been measured, simulated or is still pending, and the acceptance records hold each test's setup and results.
For timings, see the control-plane measurement record, with hardware, baselines and limits for each observation.
Follow the roadmap
x-eugene-plexus-ignored-settings header. Web search is run by your install, on a Brave, Google Search or SearXNG account; only the search terms reach the search service. Claude Code, Codex and Open WebUI have run with actual local models. This is not full vendor API parity.top_k and structured output are carried. thinking decides whether reasoning comes back as thinking blocks; context_management and cache hints are accepted but not enforced. A document by URL or file id, citations, MCP servers and server-side tools other than web search are refused, in the words Claude Code looks for to carry on without them. See the gateway contract for the complete mapping.