All work

Case study ·

Reusable agent platform

A self-hosted platform that runs AI agents for any use case behind one LLM gateway, with versioned skills, per-user access control and human approval on risky actions.

  • 32MCP tools
  • 16services, one compose file
  • Python
  • FastAPI
  • LangGraph
  • MCP
  • Bifrost
  • PostgreSQL
  • pgvector
  • Keycloak
  • Langfuse
  • React
  • Docker

Context

Each new AI use case was being built as its own system: its own agent loop, its own model client, its own way of storing documents and its own idea of who may see what. The industrial reliability agent was one of these, tied to a single LLM.

Problem

Building every agent from scratch repeated the same hard parts each time: provider lock-in, access control, auditability and a safe way to let an agent act. A new use case should mean writing skills and tools, not another platform.

Role

I designed and built the platform: the gateway-first architecture, the skill registry and its access model, the tool server, and the approval and sandboxing controls.

Architecture

  1. Sessions. Users sign in through Keycloak and get their own agent sessions in a web app, with each turn streamed back over server-sent events.
  2. Agent runtime. Agents run on deepagents (LangGraph), which provides planning, sub-agents and checkpointed sessions.
  3. LLM gateway. Every LLM and embedding call goes through Bifrost, so the model is a configuration setting and provider keys never reach the agent.
  4. Tool server. One MCP server exposes the agent's capabilities as tools: skill search, document ingestion and retrieval, workflows and human review. Each tool checks the caller's permissions itself.
  5. Skills. Versioned skills live in a registry scoped to user, project or organisation. An admin approves new skills, and a skill stays invisible to anyone without a grant.
  6. Programs. Agents can save code as programs that run on demand or on a schedule in a separate sandbox, with hard limits on LLM calls, tokens and output.
  7. Data and traces. Postgres holds application data, embeddings and the full-text index, with row-level security separating organisations. File bytes live in object storage, and Langfuse traces every step.
Reusable agent platform architectureA signed-in user's session runs on an agent runtime. The runtime calls models only through an LLM gateway, acts only through an MCP tool server that checks permissions per tool, and runs saved programs in a budgeted sandbox. Postgres, object storage and Langfuse hold the data, files and traces underneath.Web appKeycloak sign-in · SSEAgent runtimedeepagents · LangGraphLLM gatewayany providerMCP toolsper-tool authProgramsbudgeted sandboxData and tracesPostgres · storage · Langfuse
Skills, documents and approvals are all reached through the tool server.

Key decisions

One gateway for every model call. Routing all LLM traffic through a gateway makes the provider a setting rather than a rewrite, and keeps credentials in one place the agent cannot read.

Authorise in the tool server, not the gateway. The gateway only governs access per key, so it cannot decide who may call which tool. Each tool checks the caller's identity and scope, and Postgres row-level security backs that up.

Skills retrieved on demand. The agent searches skill metadata and loads a skill only when it needs one, so the library can grow without growing every prompt.

Untrusted output on its own origin. Files the agent produces, including interactive HTML, render on a separate web origin, so a malicious artifact cannot reach the app or the user's session.

Results

  • The industrial reliability agent runs on the platform and is in use.
  • Accounting automation, such as balance sheet management, is being built on the same platform as a second use case.
  • Switching the model is a configuration change, not a code change.
  • A suite of invariant checks fails whenever a new tool is not classified for access, approval and trust.

What I'd change

Guardrails come next. Today the platform relies on approval gates and per-tool permissions; I'm building stronger guardrails around what agents may read, write and send to a model. I would also add unit tests to the Python services, which today rely on the invariant checks alone.