Connect · v0.2.1 · Apache-2.0

@flashyos/llm-gateway

One inference seam, several providers behind it

Provider-agnostic inference seam: adapters, automatic failover, and an empty-success guard. One package so the estate stops carrying copies.

npm install @flashyos/llm-gatewaynpm ↗source ↗

Why it exists

A provider-agnostic seam: adapters, automatic failover, and one namespace for models that live in different places. Written because inference belongs behind an interface — a system whose behaviour changes when a vendor has a bad afternoon is not a system.

How it works

`@flashyos/llm-gateway` is a provider-agnostic inference seam: adapters, automatic failover, and an empty-success guard. It takes configuration and an error factory and does not read your environment or import your error class — that boundary is what makes it a package rather than a copy with extra steps.

Cutover is configuration: `provider` selects who serves inference and the other configured provider becomes the automatic fallback, so changing providers is an environment change, never a deploy. One `call()` returns text with token counts, a request id and the provider that served it.

It exists because this code lived twice as byte-identical copies — once in ClaimYour.Gold, once in `apps/ops` — and two copies of the thing every AI call passes through drift expensively.

Related concepts

IntentMesh agentic autonomous organization machine-readable organizational trust AI agent trust register

Use it

One call, whichever provider is up

import { createGateway } from '@flashyos/llm-gateway';

const gateway = createGateway({ providers: ['anthropic', 'gatewayz'] });
const reply = await gateway.complete({ model: 'claude-sonnet-5', messages });

In practice

Survive a provider’s bad afternoon

An app configures two providers; when the primary fails, the gateway fails over to the other automatically, so a vendor outage is a fallback rather than an incident and a deploy.

Put inference behind one seam

A codebase making calls in several places routes them all through `createGateway`, so model names, failover and the empty-success guard live in one place instead of being reimplemented per call site.

How it stays honest

The seam is verified on demand from an environment that can actually reach the provider — the remote-agent containers cannot, by egress policy, which is why it is a workflow rather than a CI step. The provider key lives only in the repository secret it reads, not in code, artifacts or the workflow file, which is the custody rule enforced by where the secret physically is. Run it yourself from Actions with the optional smoke input to spend one real inference call against staging.

.github/workflows/gatewayz-verify.yml ↗the workflow that keeps this current — read it, it hides nothing

Questions

What does @flashyos/llm-gateway do?

It is a provider-agnostic inference seam: adapters for several providers behind one interface, automatic failover, and an empty-success guard. One call() returns text, token counts, a request id and the serving provider.

How do I switch inference providers?

Change the `provider` configuration value. The other configured provider becomes the automatic fallback, so switching who serves inference is an environment change, never a code change or a deploy.

Why not call a provider SDK directly?

Because the same inference code lived twice as byte-identical copies and drifted. Routing every call through one seam — which does not read your environment or import your error class — keeps the thing every AI call passes through in one place.

The licence

Apache-2.0, read from the package’s own manifest. Embed it in anything, including closed software — that is what makes a spec adoptable and a verifier worth running. FlashyOS’s server side is AGPL-3.0-only instead, and a test in the monorepo asserts the direction between them: AGPL code may consume this, this may never consume AGPL code.

Keep reading

Also in connect

@flashyos/agent @flashyos/mcp @flashyos/create-mesh-agent @flashyos/create-mesh-node @flashyos/page @flashyos/eslint-config @flashyos/wallet-wdk @flashyos/signer @flashyos/wdk