Self-Hosting AI: Why We Chose to Run in Your Infrastructure

Cut the outbound internet. Block every third-party control plane. Give the agent nothing except your LLM endpoint and your database. Which of its features stop working?

That is the question a security architect asks before they let an AI product touch production data. Most "self-hosted" pages dodge it. The word gets stretched to cover a SaaS with a customer-hosted proxy, a vendor gateway that forwards prompts on your behalf, or a shared Kubernetes namespace labelled "your VPC." When the reviewer digs, they usually find the prompts still cross the vendor's network on the vendor's key.

The rest of this post walks the mechanisms that make "runs in your infrastructure" a technical property, not a phrase on a pricing page. Four invariants carry the argument.

Four Invariants That Keep the Data Path Local

Every self-hosting claim reduces to whether the LLM request, the SQL execution, and the sandboxed code path can be forced to stay on one side of the wire. Limerence pins that with four invariants.

The next four sections walk each invariant to the code that enforces it. After that, the honest half of the post: the caveats, the traps, and the failure windows the deployer inherits once the boundary starts.

The Provider Registry Has No Fallback Path

Every agent turn needs a signed LLM request. In Limerence, that signature comes from a factory called modelProvider(provider, apiKey, baseUrl) — and there is no code path that reaches it without a resolved team-provider row loaded from the customer's own database.

The write side is a simple upsert. A team admin POSTs { apiKey, baseUrl, isEnabled } for a chosen provider slug (openai, anthropic, ollama, lmstudio, oai-compat-*). For oai-compat shapes, the handler refuses the write unless both an API key and a base URL are present. The row lives in a TeamAIProvider table keyed by (teamId, provider), and the response masks the saved key so it never round-trips back to the browser.

The read side is where the invariant sits. When an agent turn resolves its model client, the sandbox spawner queries TeamAIProvider for the row keyed by (teamId, provider), checks isEnabled, and only then calls the provider registry with (apiKey, baseUrl) from that row. If the row is missing or disabled, the code throws agent/provider-not-configured and the turn dies before the sandbox is even created. There is no callsite anywhere in the resolver that supplies an ambient key — no shared inference proxy, no "we'll cover you until you finish setup." A misconfigured deployment fails closed on the first request. That refusal is the feature.

For teams that already went through a model-committee review, this is the point where their approved architecture actually applies. A bank that got Claude on Bedrock signed off configures an OpenAI-compatible provider with the internal gateway URL and a service-account token. The backend runs in the same VPC as the gateway, so LLM traffic stays inside the VPC boundary — no traversal of the public internet, and no traversal of Limerence-owned infrastructure.

Postgres Sessions Open Read-Only at the Driver

The next invariant is what happens when the agent's SQL reaches the database. The Postgres adapter manages a per-datasource connection pool. Every connection the pool hands out runs one command before it is returned to the caller.

SET default_transaction_read_only = on;

SET default_transaction_read_only = on;

It stays in effect for the life of the connection, so any UPDATE, INSERT, DELETE, or DDL the agent might attempt fails at the database with a read-only transaction error before a single row is touched. The guarantee is not a policy layer or a prompt instruction. It is a Postgres server refusing to write because the client asked it not to.

This session-level enforcement is Postgres-specific. The BigQuery, MSSQL, MySQL, and MariaDB adapters rely on the DB user being read-only by convention — a deployment-time concern that lives at IAM or role configuration.The defense-in-depth writeup walks the per-engine story. A deployer picking one of those engines carries responsibility for provisioning the read-only role before the data source is connected.

Even against a model that occasionally produces a write, the SQL validator runs first — but the argument does not depend on stacking. The single strong guarantee is the session flag: if the validator missed a mutation, the driver would still refuse it. That is what "in-code enforcement" means as opposed to policy-layer hope.

Rootless docker-in-docker Means a Breakout Lands Unprivileged

Some agent turns produce code that has to run — a Python snippet that reshapes a result set, a small script the model wrote to compute a metric. That code executes in a per-chat sandbox container. If the sandbox host was a shared Docker daemon, a breakout would land the escaping process on the host as root. So the sandbox host is not a shared daemon.

Instead, a dedicated sandbox-runner service runs docker:27-dind-rootless. It exposes an inner Docker daemon over a fixed TCP address, and the backend is wired to talk to that address for every sandbox spawn. Because the inner dockerd runs rootless, its UID is remapped through the host's subuid range. A container that breaks out lands as an unprivileged remapped user on the runner container, not as root on the host.Rootless docker-in-docker assumes the host kernel supports user namespaces, has permissive apparmor/seccomp policies, and provides sufficient subuid/subgid ranges. On stripped-down Kubernetes nodes this may not be available; the shipped topology does not offer a rootful fallback.

The artifacts store — a longer-lived service the sandbox reads and writes into — hardens further. Its container drops every Linux capability with cap_drop: [ALL] and disables privilege escalation with no-new-privileges. The posture is not "we hope the sandbox is safe." It is "if the sandbox is compromised, the attacker inherits one unprivileged UID inside a rootless daemon, plus whatever authority the gateway agrees to grant on its behalf — and the next section names what that authority is scoped to."

The Sandbox Gateway Scopes Authority to One Agent at a Time

Sandboxed code needs to run SQL against the customer's database, but handing the sandbox a live database password would defeat the isolation model. If the sandbox is compromised, the credential is compromised too. So the sandbox never sees it — and the token it does hold is deliberately scoped so a leak does not become a standing key to every data source in the tenant.

Every sandbox spawns with two environment variables: a gateway URL pointing at the backend's internal address, and an HS256 JWT signed with SANDBOX_GATEWAY_SECRET. The token's payload names the agent and the chat. It carries no standing data access — the gateway looks up the agent's data-source allowlist on every call.

1. 1
   Spawn. The backend starts a sandbox with SANDBOX_GATEWAY_URL and a per-sandbox JWT in its environment. No database host, no database user, no database password. The JWT names only the agent and chat.
2. 2
   Request. The sandbox's sql CLI POSTs { db, sql } to the gateway as a JSON-RPC call, presenting the JWT as its bearer token.
3. 3
   Authorize. The gateway verifies the JWT signature against SANDBOX_GATEWAY_SECRET, then queries Prisma for an agent whose ID matches the token's scope and whose data-source relation includes the requested ID. If the requested db is not in the agent's live allowlist, the RPC returns a JSON-RPC forbidden error (-32003) and no connection is opened.
4. 4
   Resolve. The backend loads the DataSource row, constructs the correct adapter through the shared factory, and validates the SQL through the same preprocessor every other query goes through.
5. 5
   Execute. The adapter acquires a pooled connection — already opened read-only for Postgres — and runs the query.
6. 6
   Stream. Results flow back through the gateway to the sandbox. The credential never left the backend process, and the sandbox held only a token whose reach ends the moment the agent's data-source assignments change.

The sandbox's blast radius is the read-only view over the specific data sources currently assigned to its agent — not the tenant's whole catalog, not a persistent capability the operator has to remember to revoke. Reassign a data source and the gateway refuses the sandbox's next call to it, immediately, without rotating any secret. That property is why authority-scoping and credential-scoping are separate invariants in the resolver.

The docker-in-docker Boundary Forces Static IPs

The isolation works because Compose DNS does not resolve across the docker-in-docker boundary. That is also its worst deploy-time trap.

A container running inside the rootless inner dockerd cannot resolve backend or database by service name — those names live in the outer Compose network's DNS, which the inner daemon does not know about. Static IPs at the outer network are what let the inner containers reach anything at all.

Once you know the trap, it is easy to avoid. A deployer who does not read the comments discovers it the first time an agent tries to run a Python cell.

Provider Keys Live in Plaintext Because the Deployer Owns the Database

The provider row stores its API key as a plain TEXT column. No application-layer envelope encryption, no unwrap-on-read from a KMS. This is deliberate. In a self-hosted deployment, the database is the customer's — its encryption at rest, its IAM, its backup encryption are all controlled by the operator.

An application-layer envelope on top of a database the customer already controls does not close a threat the deployer cares about. It would encrypt the value using a key the deployer also has to manage — sourced from where? Rotated by whom? — while providing no new guarantee against an attacker who has already compromised the database or the process that decrypts the envelope. The threat model envelope encryption addresses is "the SaaS provider's DBAs can see customer secrets," and it does not apply when the DBA is the customer.

The one place this reasoning cracks is the audit checkbox. Some compliance regimes require that the value stored in a database row not be the same value used to sign an upstream API call — a control that exists precisely to survive the case where reviewers cannot verify DB-layer encryption end-to-end. For those buyers, an envelope key sourced from Secret Manager and unwrapped in-process is a real feature request, not paranoia — and the codebase does not have one today.

The deployer inherits, in exchange, the responsibility for choosing where the database lives. Managed Postgres with a customer-managed key. RDS with KMS. Cloud SQL with IAM DB auth. All the standard database-layer controls apply. The boundary that used to be "trust the vendor's crypto team" becomes "configure your database correctly." For teams evaluating self-hosting in the first place, that trade is usually the whole point.

What the Codebase Does Not Do

The four invariants describe what is enforced. The absence of specific mechanisms is worth naming just as clearly.

There is no key rotation ceremony. The write handler upserts the provider row and replaces the key immediately — no dual-key overlap window, no hashed history, no explicit revocation acknowledgement. There is no provider failover: if the primary returns 5xx, the agent fails the turn. There is no per-team usage cap; token spend lives on the provider's dashboard.

The Failure Windows the Deployer Owns

Enforcement in code stops at the edges of the process. Beyond those edges are the operational failures the codebase does not prevent — the deployer owns them by owning the environment.

Provider outage or key revocation. When a team's provider key is revoked upstream or the endpoint returns 5xx, the runtime call throws whatever the SDK throws. There is no queued-retry path, no automatic switch to a second provider. The turn fails. Recovery is "fix the key or the endpoint, then rerun."

Sandbox daemon restart. If the sandbox-runner restarts, every in-flight sandbox container dies. The backend re-establishes its Docker client against tcp://sandbox-runner:2375 when the daemon is back, and new turns spawn fresh sandboxes normally. What survives the restart is whatever was flushed to the shared sandbox-workspace volume — the artifacts store keeps that mount. What does not survive is in-memory per-chat state: a Python cell that was mid-computation errors out on the next turn, and the chat surfaces the failure. There is no automatic replay. Recovery is "the user reruns the cell against the same artifacts."

Customer DB connection storms. The Postgres adapter uses a per-datasource pool with no circuit breaker. If the customer's database caps max_connections low and a burst of agent turns arrives, pool acquisitions queue or reject at the database. The operator tunes the pool size and the DB's connection limit in tandem; nothing in the application will do it for them.

Login without HTTPS. Better-auth issues session cookies with the Secure flag, and browsers refuse to store or send those cookies over a plain-HTTP origin. The result at the UI is not a red error — it is a login flow that appears to succeed and then bounces back to the sign-in page because no session cookie came back on the next request. Backend logs show the login endpoint returning 200. Only the browser knows the cookie was dropped. The Dokploy README warns about this, but it is a runtime failure that only surfaces when the first real user signs in.

Two Deployment Shapes, One Enforcement Boundary

There is more than one way to run all this. The invariants do not care which — they are properties of the code, not the topology.

Both shapes preserve the same enforcement boundary on the data path. Both require the deployer to own TLS, secret storage, and the choice of LLM endpoint. Neither shape lets the codebase phone home for an LLM fallback that would compromise the provider invariant — updates to the deployment itself are a separate pull-based channel the operator triggers.

The question at the top of this post has a straightforward answer. Cut the outbound internet. Block every third-party control plane. Point the agent at your internal LLM endpoint and your database. It still works, because the code is already assuming that is the deployment.