Skip to content
/
Tier B A11 Governance

AgentGateway

agentgateway · agentgateway/agentgateway · ★ 2.9k · last commit 2026-05-26

Primitive shape
No installable primitives
00

Summary

AgentGateway — Summary

AgentGateway is a Linux Foundation project — an open-source, Rust-based agentic proxy built on AI-native protocols (MCP and A2A) providing drop-in security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication across any framework and environment. It serves as a unified connectivity layer with four major subsystems: LLM Gateway (multi-provider routing with budget controls), MCP Gateway (tool federation with OAuth and OpenAPI integration), A2A Gateway (secure agent-to-agent communication), and Inference Routing (Kubernetes-aware GPU/model routing). Guardrails are multi-layered (regex, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor). The built-in UI runs at localhost:15000/ui.

Compared to seeds: AgentGateway is closest in spirit to ContextForge (both are MCP gateways with observability) but is Rust-based, has native Kubernetes controller + Gateway API support, includes a dedicated LLM gateway with budget controls and an A2A layer, and ships as a single compiled binary — dramatically simpler to deploy than ContextForge's Python stack. Unlike all 11 seeds, AgentGateway is a network-layer proxy, not an agent methodology or harness.

01

Overview

AgentGateway — Overview

Origin

Linux Foundation project. Rust-based, Apache-2.0 licensed. 2,871 stars, 481 forks as of 2026-05-26. Active development with community meeting recordings.

Philosophy

"The first complete connectivity solution for Agentic AI."

AgentGateway's position: as AI agents proliferate, the connectivity layer (agent-to-tool, agent-to-LLM, agent-to-agent) needs the same governance primitives that API gateways brought to microservices — but with AI-native protocol support (MCP, A2A).

Design Goals

  1. Protocol-native: MCP and A2A as first-class protocols (not an afterthought adapter layer)
  2. Language-agnostic: Any AI framework/agent connects via standard protocols
  3. Drop-in: Should require minimal changes to existing agent code
  4. Kubernetes-native: First-class Kubernetes controller with Gateway API integration for cloud deployments
  5. Observable: OpenTelemetry metrics, logs, and tracing out of the box

Key Features

  • LLM Gateway: Unified OpenAI-compatible API across OpenAI, Anthropic, Gemini, Bedrock with budget/spend controls, prompt enrichment, load balancing, and failover
  • MCP Gateway: Tool federation with stdio/HTTP/SSE/Streamable HTTP transports, OpenAPI integration, OAuth auth
  • A2A Gateway: Capability discovery, modality negotiation, task collaboration between agents
  • Inference Routing: Kubernetes Inference Gateway integration with GPU utilization, KV cache, LoRA adapter awareness
  • Guardrails: Regex, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor, custom webhooks
  • CEL policy engine: Fine-grained RBAC using Common Expression Language

Linux Foundation Governance

Part of LF's portfolio alongside Kubernetes and other infrastructure projects. Community meetings with published recordings and Google Calendar.

02

Architecture

AgentGateway — Architecture

Distribution

  • Binary: Single Rust binary (cross-platform, builds with cargo build)
  • Docker: Dockerfile and Dockerfile.windows in repo root
  • Kubernetes: Built-in controller + Gateway API support (Tiltfile for development)

Build from Source

cd ui && npm install && npm run build
cargo build --release
./target/release/agentgateway

UI available at http://localhost:15000/ui

Directory Structure

crates/
  agentgateway/         # Main gateway binary
  agentgateway-app/     # Application shell
  cel-fork/             # CEL policy engine fork
  celx/                 # CEL extensions
  core/                 # Core types and abstractions
  hbone/                # HBONE tunnel support
  htpasswd-verify-fork/ # htpasswd auth
  pool/                 # Connection pooling
  protos/               # Protocol buffers
  xds/                  # xDS API support (Envoy-compatible)
ui/                     # Next.js frontend (built into binary)
api/                    # API definitions (Go: resource.pb.go)
architecture/           # Architecture diagrams
schema/                 # JSON/YAML configuration schemas
manifests/              # Kubernetes manifests
controller/             # Kubernetes controller
design/                 # Design documents
tools/                  # Development tools

Required Runtime

  • Rust 1.86+ (for building from source)
  • npm 10+ (for UI build)
  • Docker (optional)
  • Kubernetes + kubectl (for k8s deployment)

Configuration

YAML-based configuration file defining listeners, backends, auth, guardrails, and routing policies.

Target AI Tools

Any MCP-compatible client and any A2A-compatible agent. Protocol-agnostic at the application level.

Transports Supported

MCP: stdio, HTTP, SSE, Streamable HTTP
LLM: OpenAI-compatible API (all major providers)
A2A: HTTP with capability negotiation
Inference: Kubernetes Inference Gateway protocol

03

Components

AgentGateway — Components

Gateway Subsystems

Subsystem Purpose
LLM Gateway Unified OpenAI-compatible API; multi-provider routing with budget/spend controls, prompt enrichment, load balancing, failover
MCP Gateway Tool federation; stdio/HTTP/SSE/Streamable HTTP transports; OpenAPI integration; OAuth authentication
A2A Gateway Agent-to-agent communication; capability discovery; modality negotiation; task collaboration
Inference Routing Kubernetes Inference Gateway; GPU utilization, KV cache, LoRA adapter, queue depth routing

Security & Governance

Component Purpose
JWT/API key auth Authentication for incoming requests
OAuth Authentication to upstream MCP servers
CEL policy engine Fine-grained RBAC using Common Expression Language (cel-fork, celx crates)
Rate limiting Per-user/team rate limits
TLS mTLS and TLS termination

Guardrails

Guardrail Purpose
Regex filter Pattern-based content filtering
OpenAI Moderation API LLM-based content classification
AWS Bedrock Guardrails AWS-managed safety policies
Google Model Armor Google-managed safety policies
Custom webhooks User-defined guardrail endpoints

Observability

  • OpenTelemetry: metrics, logs, and distributed tracing
  • Built-in UI with architecture visualization at :15000/ui

Kubernetes Components

Component Purpose
Controller Kubernetes controller for declarative gateway management
Gateway API Standard Kubernetes Gateway API integration
Inference Gateway Kubernetes-aware GPU/model routing extensions
Manifests Kubernetes deployment manifests in manifests/

Protocol Buffers

api/resource.pb.go — protobuf definitions for gateway resources. buf.gen.yaml and buf.yaml for protobuf generation.

Development Tools

  • Tiltfile — Tilt dev environment with hot reload
  • Makefile — build targets
  • .githooks/ — pre-commit hooks (from .githooks/)
05

Prompts

AgentGateway — Prompts

AgentGateway is a Rust-based network proxy, not a prompting framework. It does not ship agent instruction prompt files. The CHARTER.md and CONTRIBUTION.md serve as the closest governance documents.

CHARTER.md (Project Charter)

The CHARTER.md defines the project's scope and governance as a Linux Foundation project. It documents the mission, governance model, and IP policies. Not a prompt file — a governance document.

DEVELOPMENT.md (Developer Instructions)

# Quickstart (GitHub, no local install)

1. Click **Code → Create codespace on main**.
2. In the terminal:
   cargo fmt --all
   cargo clippy --all -- -D warnings
   cargo test --all
3. If you touched the UI:
   cd ui
   npm ci
   npm test

# Local Development

Requirements:
- Rust 1.86+
- npm 10+

Build the agentgateway UI:
cd ui
npm install
npm run build

Build the agentgateway binary:
export CARGO_NET_GIT_FETCH_WITH_CLI=true
make build

Run the agentgateway binary:
./target/release/agentgateway

Open your browser and navigate to `http://localhost:15000/ui` to see the agentgateway UI.

Prompting technique: Step-by-step procedural instructions for AI coding assistants — minimal prose, maximum precision. The three-command Codespace quickstart optimizes for developer time-to-first-contribution.

CEL Policy Language (Runtime Prompts)

AgentGateway's "prompts" for behavior control are CEL (Common Expression Language) policy expressions:

# Example: Allow only finance-cleared agents to access finance tools
principal in Group::"finance-cleared" &&
resource.namespace == "finance" &&
context.request_time.getHours() >= 9 &&
context.request_time.getHours() <= 17

Prompting technique: Declarative policy expressions — the operator defines allowed behaviors using a structured expression language, not natural language prompts. This is deterministic, not probabilistic.

Guardrail Configuration (Runtime Behavior)

guardrails:
  - type: regex
    patterns:
      - "\\b(SSN|social security)\\b"
    action: block
  - type: openai-moderation
    categories: [hate, violence]
    threshold: 0.8
  - type: bedrock-guardrails
    guardrail_id: "my-guardrail-id"

Prompting technique: Configuration-as-code for safety policies. Deterministic rule evaluation, not LLM-based judgment.

09

Uniqueness

AgentGateway — Uniqueness & Positioning

Differs from Seeds

AgentGateway is categorically different from all 11 seeds — it is a network-layer proxy for AI protocols, not a coding methodology or agent harness. The most comparable seed is the MCP-anchored archetype (taskmaster-ai, ccmemory, claude-flow) but those embed MCP servers inside coding tools; AgentGateway proxies MCP traffic at the network edge. Compared to ContextForge (same batch, also MCP gateway), AgentGateway is Rust-based (lower latency, single binary), includes a dedicated LLM gateway with multi-provider routing, has native Kubernetes controller support, and integrates with Kubernetes Inference Gateway for GPU-aware model routing — capabilities that position it as the more cloud-native, performance-oriented option.

Unique Characteristics

  1. Rust single binary: Dramatically simpler deployment than Python (ContextForge) or TypeScript (Archestra) stacks. No runtime dependencies beyond the binary.
  2. Kubernetes Inference Routing: Only framework in corpus with hardware-aware LLM routing (GPU utilization, KV cache, LoRA adapters, queue depth). Bridges the gap between AI gateway and Kubernetes scheduler.
  3. Linux Foundation governance: Vendor-neutral, formal CNCF-track governance. Same foundation as Kubernetes.
  4. CEL policy engine: Custom fork (cel-fork) for fine-grained RBAC with deterministic expression evaluation — more expressive than JWT scopes alone.
  5. Multi-guardrail stack: Only framework integrating AWS Bedrock Guardrails AND Google Model Armor AND OpenAI Moderation AND regex AND custom webhooks in a single product.
  6. A2A Gateway: Dedicated agent-to-agent protocol layer with capability discovery and modality negotiation — explicitly designed for multi-agent architectures.
  7. xDS control plane support: Envoy-compatible dynamic configuration updates — allows integration with existing service mesh control planes.

Positioning

"The Envoy/Istio for AI agents." Targets teams building production multi-agent systems on Kubernetes who need the same governance primitives they use for microservices, adapted for AI-native protocols.

Observable Failure Modes

  • Rust build complexity: Requires Rust toolchain + npm for UI — higher contributor barrier than Python alternatives.
  • Stateless budget tracking: If spend counters are in-memory, they reset on restart — budget enforcement may be leaky without external state.
  • Policy complexity: CEL expressions are powerful but require expertise; misconfigured policies could silently permit or block legitimate requests.
  • Early-stage A2A: A2A protocol itself is new; the A2A Gateway's production readiness is unverified.

Cross-References

  • Competes with: ContextForge (IBM), Archestra, Plano — all MCP gateway/proxy frameworks
  • Linux Foundation member: same foundation as Archestra/CNCF
  • CEL: same policy language as Kubernetes RBAC policies
04

Workflow

AgentGateway — Workflow

AgentGateway is an infrastructure proxy. Its workflow is deployment and configuration, not agent task execution.

Standalone Deployment

Phase Artifact
Build/install cargo build --release or docker pull
Configure YAML config file with listeners, backends, guardrails
Start ./target/release/agentgateway --config config.yaml
Connect clients Point MCP/A2A clients at gateway endpoint

Kubernetes Deployment

Phase Artifact
Install controller kubectl apply -f manifests/
Create Gateway resource Kubernetes Gateway API manifest
Define backends MCP server backend configurations
Apply policies CEL RBAC policies, guardrail configs

LLM Request Flow

  1. Agent sends OpenAI-compatible request to LLM Gateway
  2. JWT/API key authentication
  3. CEL policy evaluation (is this agent allowed this model/action?)
  4. Budget check (remaining spend?)
  5. Load balancing across configured providers
  6. Guardrail evaluation (pre-request)
  7. Request forwarded to provider
  8. Guardrail evaluation (post-response)
  9. Response returned with OTel trace

MCP Tool Call Flow

  1. AI client sends MCP JSON-RPC to MCP Gateway
  2. Authentication (JWT, API key, or OAuth)
  3. Tool routing to registered upstream MCP server
  4. Response returned via configured transport (SSE, HTTP, etc.)

A2A Agent Discovery Flow

  1. Agent queries A2A Gateway for capability discovery
  2. Gateway returns registered agent capabilities
  3. Task collaboration routed through gateway
  4. Results returned with full trace

Approval Gates

None — policy enforcement is automatic via CEL rules, not interactive.

06

Memory Context

AgentGateway — Memory & Context

State Storage

AgentGateway is primarily a stateless proxy. State exists for:

  1. Configuration: YAML config file loaded at startup (backends, policies, guardrails)
  2. Budget tracking: In-memory or persistent spend counters per agent/team
  3. Rate limit counters: Per-connection rate limit state
  4. Connection pools: pool crate manages upstream connection pools

No SQL database is bundled. State is minimal by design — the gateway forwards requests, it does not accumulate conversation history.

Session State

No persistent session storage for agent conversations. Each request is stateless. The upstream MCP servers or LLMs maintain any necessary session context.

Memory Persistence

  • Configuration: persistent (YAML file)
  • Budget/spend counters: unknown (likely in-memory; would reset on restart)
  • Request traces: exported to external OTel backends — not stored locally

Context Compaction

Not applicable. AgentGateway does not manage LLM context windows.

Cross-Session Handoff

Not applicable at the gateway level. The gateway is transparent to session state managed by connected agents and LLMs.

xDS / Control Plane

The xds/ crate implements xDS API support (Envoy-compatible control plane protocol). This enables dynamic configuration updates without restarts — the gateway can receive updated policy/routing rules from a control plane at runtime.

Kubernetes State

In Kubernetes deployments, the controller maintains Gateway resource state in the Kubernetes API server (etcd-backed). This provides durable configuration and supports declarative updates.

07

Orchestration

AgentGateway — Orchestration

Multi-Agent Support

Yes — through the A2A Gateway. AgentGateway enables agent-to-agent communication with capability discovery and task collaboration. However, it is a router for agent interactions, not an orchestrator that sequences agent tasks.

Orchestration Pattern

None at the task level. AgentGateway routes requests; orchestration logic lives in the agents themselves.

Execution Mode

Background daemon — the gateway runs as a persistent network service.

Multi-Model Routing

Yes — the LLM Gateway supports:

  • Load balancing across multiple providers/models
  • Failover when primary provider is unavailable
  • Inference Routing for Kubernetes-hosted models: GPU utilization, KV cache, LoRA adapter, queue depth
  • Budget-based routing: route to cheaper models when budget is low

Model Role Mapping

No fixed role assignments. The LLM Gateway routes based on:

  • Provider capability (model name)
  • Budget constraints (spend thresholds)
  • Kubernetes inference metrics (for self-hosted models)

Isolation Mechanism

None at the tool execution level. Each MCP tool call is proxied to its upstream server; isolation is the server's responsibility.

A2A Orchestration

The A2A Gateway enables:

  • Capability discovery: agents announce what they can do
  • Modality negotiation: text, vision, audio capability matching
  • Task collaboration: multi-agent task decomposition (routing, not execution)

CEL Policy Engine

The CEL-based RBAC (cel-fork, celx crates) evaluates policies at request time. This is deterministic policy enforcement, not AI-based routing decisions.

Inference Routing Details

Kubernetes Inference Gateway extensions route LLM requests based on:

  • GPU utilization metrics
  • KV cache occupancy
  • LoRA adapter availability
  • Queue depth

This is the most sophisticated routing layer in the corpus — actual hardware-aware scheduling.

Consensus Mechanism

None. Single-instance or horizontally-scaled stateless instances.

08

Ui Cli Surface

AgentGateway — UI/CLI Surface

CLI Binary

Name: agentgateway Type: Standalone Rust binary (not a thin wrapper) Build: cargo build --release && ./target/release/agentgateway Subcommands: unknown (CLI flags from YAML config file)

Local Web Dashboard

Exists: Yes — bundled into the binary Type: Web dashboard Port: 15000 URL: http://localhost:15000/ui Tech Stack: Next.js (in ui/ directory, compiled and embedded into Rust binary)

Features (from README screenshot description):

  • Architecture visualization showing agent-to-agent and agent-to-tool connections
  • Gateway status and backend health
  • Request/response inspection
  • Multi-provider LLM configuration

Kubernetes UI

In Kubernetes deployments, standard Kubernetes tooling (kubectl, Lens, k9s) provides operational visibility. The controller integrates with Kubernetes Gateway API for declarative management.

Observability

  • OpenTelemetry: Full OTel integration — metrics, logs, distributed tracing
  • Metrics: Prometheus-compatible endpoint
  • Traces: Exportable to any OTel-compatible backend (Jaeger, Zipkin, etc.)
  • Logs: Structured logging

API Surface

  • LLM Gateway: OpenAI-compatible API endpoint (drop-in replacement)
  • MCP Gateway: MCP JSON-RPC endpoint (multiple transports)
  • A2A Gateway: A2A protocol endpoint
  • Admin API: Gateway configuration and management

Development UX

# Dev with hot reload (Tilt)
tilt up

# Build
make build

# Test
cargo test --all

# Format + lint
cargo fmt --all
cargo clippy --all -- -D warnings

IDE Integration

None — AgentGateway is infrastructure, not an IDE plugin. Developers interact via YAML config files and the web UI.

Related frameworks

same archetype · same primary tool · same memory type

OpenHarness ★ 13k
A11 Governance

Open-source Python agent runtime providing complete harness infrastructure: tools, memory, governance, swarm coordination, and…

Trae Agent ★ 12k
A11 Governance

Research-friendly open-source CLI coding agent by ByteDance, designed for academic ablation studies and modular LLM provider…

Sweep AI ★ 7.7k
A11 Governance

Autonomous GitHub bot that converts issues to pull requests using a sequential multi-agent pipeline.

Agent Governance Toolkit (microsoft) ★ 2.3k
A11 Governance

Enterprise-grade AI agent governance: YAML policy enforcement, 12-vector prompt injection defense, zero-trust identity,…

TDD Guard ★ 2.1k
A11 Governance

Mechanically enforces the Red-Green-Refactor TDD cycle by blocking file writes that violate TDD principles via a PreToolUse hook…

Agentic Coding Flywheel Setup (ACFS) ★ 1.5k
A11 Governance

Take a complete beginner from laptop to three AI coding agents running on a VPS in 30 minutes via an idempotent manifest-driven…