Architecture
Eugene Plexus installs and manages engines, helps choose model settings for your hardware, and connects compatible tools to local and cloud backends. One machine or a homelab, the same layers do the work. It is not an inference engine or a chat app.
Alpha.2 snapshot: . Current alpha available for testing. Current boundaries.
01 / The layers
Inference flows through the gateway and a driver. Beside that path, the agent supervises engines, the library prepares models, and the control root maintains the install. Discovery, key checks, model defaults and a model wake can still depend on those services.
Supported chat, tool-call and embedding requests, with revocable client keys. Compatibility has limits.
One per install. The only thing your tools need to know about.
A thin adapter in front of each backend. It translates and reports capabilities.
llama.cpp is installed and supervised; vLLM is user-installed and supervised. External servers stay under your control. CLI backends invoke your installed tools using their existing login.
Local engines use your machines. Cloud providers run on their own hardware.
02 / The components
Each has its own repository. The four service kinds are gateway, inference driver, library and control root; the agent supervises them, and the UI is a static export it serves. Six consumers generate types from pinned OpenAPI 3.1 contracts in the specs repository. Components share schemas, not code.
The process on each machine. It installs and updates llama.cpp, supervises components and engines, checks memory before launch, serves the web UI, and optionally keeps local copies of the models this node runs.
Talks to: Local components and engines, the control root for enrollment and discovery, and other agents for console requests.
SourceOne address serving OpenAI-compatible chat and embeddings plus Anthropic Messages. Both chat formats share routing, replica balancing, ordered failover, wake-on-demand, and retained request metrics.
Talks to: Your tools on one side; every inference driver on the other. Reads model defaults from the Library through its agent.
SourceA small adapter in front of one backend: a llama.cpp or vLLM runtime, an Ollama or LM Studio server, a cloud API, or a subscription CLI. It translates requests and reports what the backend can do.
Talks to: The gateway above it; one backend below it.
SourceYour model folders, with GGUF and safetensors scanning, named profiles for launch and generation settings, catalogue search and resumable downloads. It suggests a starter model and estimates memory fit at a stated context length for the selected machine.
Talks to: The disk, the model hub, agents launching models, and gateways reading model defaults.
SourceThe trust root, on a single machine or across a homelab. It maintains the node registry and replicated control state, issues one-time join tokens, coordinates signed rekeying, and can have warm standbys.
Talks to: Every machine’s agent. It spawns nothing itself.
SourceHome for setup and a first reply; a resource tree for the whole install. Includes model discovery, downloads and loading in a tasks tray, actionable issues, remote folder browsing, client keys, metrics, and a diagnostic playground. Static files, not a separate server.
Talks to: Its own agent, which forwards to the right component wherever it runs.
Source03 / One request
The whole point of the layering: your tool never has to know which machine, which engine, or which model file. It asks the gateway, and the answer says where it went.
An editor or coding client sends OpenAI-style chat or Anthropic Messages with a client key. The Messages route accepts a bearer token or x-api-key; OpenAI routes use bearer tokens.
It resolves the model, checks readiness, wakes it if needed, and balances replicas. Omitted generation settings come from that model’s default profile, then gateway settings. Eligible backend failures can move the request to the next configured tier.
One driver turns the request into its backend protocol and reports capabilities. Token streaming depends on that backend; Codex CLI returns whole messages rather than individual tokens.
llama.cpp, vLLM, Ollama, or a cloud provider does the actual work on the hardware it has.
Metrics retain the serving backend, timing, tier, and failed attempts for both chat formats. OpenAI replies include routing metadata in the body or final stream frame; Anthropic keeps its own body format and uses headers.
One rule worth knowing: failover happens before the first token. Once an answer has started streaming, the gateway will not splice a second model onto it. A cut-off answer is reported as cut off, never quietly completed by someone else.
Not every failure is retryable: a deadline while a backend is still computing returns a timeout, not a second computation on another replica. An unreachable host can still fail over. Embeddings fail over only between replicas of the same model, never into a different vector space.
04 / One computer, or several
There is one shape, not two products. On a single PC every component runs beside the engines. Across machines, the control plane moves to wherever is convenient and each GPU box keeps only what needs the hardware.
Everything on one box. The browser opens the agent’s address and the gateway’s port is the one your tools use.
Each machine runs an agent. A Library folder records its mounts once; nodes can override the path and optionally keep a local copy. Tools use one gateway, and any agent’s console can manage the enrolled nodes.
05 / What stays where
In the directories you point the library at, with plain names. Downloads land there too. No content-addressed cache, no hidden store. Remove Eugene Plexus and the files are still there.
Originals stay yours; local copies are optional.Operator sessions administer the install; named client keys allow inference and can be revoked. Nodes enroll with one-time join tokens, then sign address announcements and verify signed rekeys. With Ed25519 signing, agents and control roots hold signing authority; gateways, libraries and drivers receive public verification keys.
Trusted hosts and a LAN or mesh VPN, not public exposure.llama.cpp is fetched, verified and kept in versioned directories by the agent. vLLM is one you install; the agent finds and drives it. Neither is forked or replaced.
Upstream, supervised.Optional local copies: a node copies only model files its own declared runtimes use, into a separate directory with a minimum-free-space setting. It manages those copies, not the source files in your Library folders. The first copy takes time; later starts can avoid reading the model over the network again.
06 / Current boundaries
As of September 21, 2026. Alpha.2 is available for early testing. There is no stable release yet. Versioned installers and a matching container image are available; this is not a production-readiness claim. Installation guide.
The acceptance records document the test setups, observed behavior and limits of what has been verified.
For timings, see the control-plane measurement record, with hardware, baselines and limits for each observation.
Follow the roadmap
top_k are rejected. thinking, context_management and cache hints are accepted but not enforced; responses identify ignored controls. OpenAI chat separately supports bounded inline PNG/JPEG input on confirmed vision backends. See the gateway contract for the complete mapping.