Skip to content

Instantly share code, notes, and snippets.

@rvennam

rvennam/001-set-up-oss-agentgateway.md

Created March 12, 2026 16:48
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save rvennam/b127efe72aa935630db826d672809b6d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save rvennam/b127efe72aa935630db826d672809b6d to your computer and use it in GitHub Desktop.
Download ZIP
OSS Agentgateway Workshop - Install, OpenAI Routing, and MCP Connectivity
Raw
001-set-up-oss-agentgateway.md

Install Agentgateway (Open Source)

In this workshop, you'll deploy the open-source Agentgateway and complete hands-on labs that showcase routing, observability, and Gen AI features.

Pre-requisites

  • Kubernetes > 1.30
  • Kubernetes Gateway API
  • helm package manager installed

Lab Objectives

  • Configure Kubernetes Gateway API CRDs
  • Configure Agentgateway CRDs
  • Install Agentgateway Controller
  • Configure agentgateway
  • Validate that components are installed

Kubernetes Gateway API CRDs

Installing the Kubernetes Gateway API custom resources is a pre-requisite to using Agentgateway.

kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml

To check if the Kubernetes Gateway API CRDs are installed:

kubectl api-resources --api-group=gateway.networking.k8s.io

Expected Output:

NAME                 SHORTNAMES   APIVERSION                           NAMESPACED   KIND
backendtlspolicies   btlspolicy   gateway.networking.k8s.io/v1         true         BackendTLSPolicy
gatewayclasses       gc           gateway.networking.k8s.io/v1         false        GatewayClass
gateways             gtw          gateway.networking.k8s.io/v1         true         Gateway
grpcroutes                        gateway.networking.k8s.io/v1         true         GRPCRoute
httproutes                        gateway.networking.k8s.io/v1         true         HTTPRoute
referencegrants      refgrant     gateway.networking.k8s.io/v1         true         ReferenceGrant
tcproutes                         gateway.networking.k8s.io/v1alpha2   true         TCPRoute
tlsroutes                         gateway.networking.k8s.io/v1         true         TLSRoute
udproutes                         gateway.networking.k8s.io/v1alpha2   true         UDPRoute

Install Agentgateway

Configure Required Variables

export AGW_VERSION=v2.2.1

Agentgateway CRDs

kubectl create namespace agentgateway-system
helm upgrade -i --create-namespace --namespace agentgateway-system \
    --version $AGW_VERSION agentgateway-crds \
    oci://ghcr.io/kgateway-dev/charts/agentgateway-crds

To check if the Agentgateway CRDs are installed:

kubectl api-resources | awk 'NR==1 || /agentgateway\.dev/'

Expected output:

NAME                                SHORTNAMES        APIVERSION                                NAMESPACED   KIND
agentgatewaybackends                agbe              agentgateway.dev/v1alpha1                 true         AgentgatewayBackend
agentgatewayparameters              agpar             agentgateway.dev/v1alpha1                 true         AgentgatewayParameters
agentgatewaypolicies                agpol             agentgateway.dev/v1alpha1                 true         AgentgatewayPolicy

Install Agentgateway Controller

Using Helm:

helm upgrade -i -n agentgateway-system agentgateway \
    oci://ghcr.io/kgateway-dev/charts/agentgateway \
    --create-namespace \
    --version $AGW_VERSION \
    -f -<<EOF
# --- Override the default Agentgateway parameters used by this GatewayClass
# If the referenced parameters are not found, the controller will use the defaults
gatewayClassParametersRefs:
  agentgateway:
    group: agentgateway.dev
    kind: AgentgatewayParameters
    name: agentgateway-config
    namespace: agentgateway-system
EOF

Check that the Agentgateway Controller is now running:

kubectl get pods -n agentgateway-system -l app.kubernetes.io/name=agentgateway

Expected Output:

NAME                            READY   STATUS    RESTARTS   AGE
agentgateway-5fc9d95758-n8vvb   1/1     Running   0          87s

Verify the GatewayClass was created:

kubectl get gatewayclass agentgateway

Deploy Agentgateway

kubectl apply -f- <<'EOF'
---
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayParameters
metadata:
  name: agentgateway-config
  namespace: agentgateway-system
spec:
  logging:
    level: info
  service:
    spec:
      type: LoadBalancer
  #--- Use rawConfig to inline custom configuration ---
  rawConfig:
    config:
      logging:
        fields:
          add:
            # --- Capture the claims from a verified JWT token if JWT policy is enabled
            jwt: 'jwt'
            # --- Capture a single request header by name (example: x-foo)
            x-foo: 'request.headers["x-foo"]'
            # --- Capture entire request body and parse it as JSON
            request.body: json(request.body)
            # --- Capture entire response body and parse it as JSON
            response.body: json(response.body)
            # --- Capture a field in the request body
            request.body.modelId: json(request.body).modelId
        format: json
      tracing:
        otlpProtocol: grpc
        otlpEndpoint: http://tempo-distributor.monitoring.svc.cluster.local:4317
        randomSampling: 'true'
        fields:
          add:
            gen_ai.operation.name: '"chat"'
            gen_ai.system: "llm.provider"
            gen_ai.prompt: 'llm.prompt'
            gen_ai.request.model: "llm.requestModel"
            gen_ai.response.model: "llm.responseModel"
            gen_ai.usage.completion_tokens: "llm.outputTokens"
            gen_ai.usage.prompt_tokens: "llm.inputTokens"
            gen_ai.request: 'flatten(llm.params)'
            # --- Capture the claims from a verified JWT token if JWT policy is enabled
            jwt: 'jwt'
            # --- Capture entire response body and parse it as JSON
            response.body: 'json(response.body)'
            # --- Capture a single request header by name (example: x-foo)
            x-foo: 'request.headers["x-foo"]'
  deployment:
    spec:
      replicas: 2
      template:
        spec:
          containers:
          - name: agentgateway
            resources:
              requests:
                cpu: 300m
                memory: 128Mi
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway-proxy
  namespace: agentgateway-system
spec:
  gatewayClassName: agentgateway
  listeners:
    - name: http
      port: 8080
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: All
EOF

Check that the Agentgateway proxy is now running:

kubectl get pods -n agentgateway-system

Expected Output:

NAME                                     READY   STATUS    RESTARTS   AGE
agentgateway-proxy-7d4c8c4d4b-lvdsq      1/1     Running   0          11m
agentgateway-proxy-9f8e7d6c5b-xkpqr      1/1     Running   0          11m
agentgateway-5f9c5b95b4-gjblt            1/1     Running   0          11m

Configure Basic Routing to OpenAI

Configure Required Variables

Replace with a valid OpenAI API key

export OPENAI_API_KEY=$OPENAI_API_KEY

Create openai api-key secret

kubectl create secret generic openai-secret -n agentgateway-system \
--from-literal="Authorization=Bearer $OPENAI_API_KEY" \
--dry-run=client -oyaml | kubectl apply -f -

Create openai route and backend

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: openai
  namespace: agentgateway-system
spec:
  parentRefs:
    - name: agentgateway-proxy
      namespace: agentgateway-system
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /openai
      backendRefs:
        - name: openai-all-models
          group: agentgateway.dev
          kind: AgentgatewayBackend
      timeouts:
        request: "120s"
---
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: openai-all-models
  namespace: agentgateway-system
spec:
  ai:
    provider:
      openai: {}
  policies:
    auth:
      secretRef:
        name: openai-secret
EOF

Curl OpenAI

export GATEWAY_IP=$(kubectl get svc -n agentgateway-system --selector=gateway.networking.k8s.io/gateway-name=agentgateway-proxy -o jsonpath='{.items[*].status.loadBalancer.ingress[0].ip}{.items[*].status.loadBalancer.ingress[0].hostname}')

curl -i "$GATEWAY_IP:8080/openai" \
  -H "content-type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Whats your favorite poem?"
      }
    ]
  }'

View Access Logs

AgentGateway automatically logs detailed information about LLM requests to stdout:

kubectl logs deploy/agentgateway-proxy -n agentgateway-system --tail 1

View Metrics Endpoint

AgentGateway exposes Prometheus-compatible metrics at the /metrics endpoint:

kubectl port-forward -n agentgateway-system deployment/agentgateway-proxy 15020:15020 & \
sleep 1 && curl -s http://localhost:15020/metrics && kill $!

OpenAI Cleanup

kubectl delete httproute -n agentgateway-system openai
kubectl delete agentgatewaybackend -n agentgateway-system openai-all-models
kubectl delete secret -n agentgateway-system openai-secret

Configure Route to MCP Server

Deploy an MCP Server

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-website-fetcher
  namespace: agentgateway-system
spec:
  selector:
    matchLabels:
      app: mcp-website-fetcher
  template:
    metadata:
      labels:
        app: mcp-website-fetcher
    spec:
      containers:
      - name: mcp-website-fetcher
        image: ghcr.io/peterj/mcp-website-fetcher:main
        imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: mcp-website-fetcher
  namespace: agentgateway-system
  labels:
    app: mcp-website-fetcher
spec:
  selector:
    app: mcp-website-fetcher
  ports:
  - port: 80
    targetPort: 8000
    appProtocol: agentgateway.dev/mcp
EOF

Create Backend and HTTPRoute

kubectl apply -f - <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: mcp-backend
  namespace: agentgateway-system
spec:
  mcp:
    targets:
    - name: mcp-target
      static:
        host: mcp-website-fetcher.agentgateway-system.svc.cluster.local
        port: 80
        protocol: SSE
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp
  namespace: agentgateway-system
spec:
  parentRefs:
  - name: agentgateway-proxy
  rules:
    - backendRefs:
      - name: mcp-backend
        group: agentgateway.dev
        kind: AgentgatewayBackend
EOF

Get Gateway IP

export GATEWAY_IP=$(kubectl get svc -n agentgateway-system --selector=gateway.networking.k8s.io/gateway-name=agentgateway-proxy -o jsonpath='{.items[*].status.loadBalancer.ingress[0].ip}{.items[*].status.loadBalancer.ingress[0].hostname}')

echo $GATEWAY_IP

Run the MCP Inspector

npx @modelcontextprotocol/inspector@0.16.2

In the MCP Inspector menu, connect to your agentgateway:

  • Transport Type: Select Streamable HTTP
  • URL: Enter http://$GATEWAY_IP:8080/mcp (replace with your actual IP)
  • Click Connect

Fetch a Website

  • From the menu bar, click the Tools tab. Then from the Tools pane, click List Tools and select the fetch tool.
  • From the fetch pane, in the url field, enter a website URL, such as https://lipsum.com/, and click Run Tool.
  • Verify that you get back the fetched URL content.

MCP Cleanup

kubectl delete httproute -n agentgateway-system mcp
kubectl delete agentgatewaybackend -n agentgateway-system mcp-backend
kubectl delete deployment -n agentgateway-system mcp-website-fetcher
kubectl delete service -n agentgateway-system mcp-website-fetcher
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment