joelhooks/00-index.md

Last active January 17, 2026 22:37

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/joelhooks/6b91831b1f801278ce9414c1f6f5b04d.js"></script>
Save joelhooks/6b91831b1f801278ce9414c1f6f5b04d to your computer and use it in GitHub Desktop.

Download ZIP

Agent-first support platform architecture spec

Raw

00-index.md

Support Platform PRD (Overview)

Agent-first customer support platform with human-in-the-loop for Skill Recordings products.

Purpose

This is the overview and index. Start here. Detailed implementation instructions live in phase and reference docs.

Vision

Every support interaction is an opportunity for an agent to help. The agent handles the high-volume, brain-dead stuff automatically. Humans approve edge cases via Slack or dashboard. Front remains the source of truth for conversations, but the agent is the brain.

Success Criteria

Reduce human touches by 80%
Sub-minute response time (draft within 60s)
Full traceability (every decision/action/approval logged)
Multi-app robustness (adding a new app is a skill init away)

Locked Decisions (Summary)

Inbox model: one inbox per product
Workflow engine: Inngest only
Vector search: Upstash defaults (hybrid, hosted embeddings)
Auth: BetterAuth
Database: PlanetScale
Webhook signing: HMAC-SHA256, Stripe-style, 5-minute replay, key rotation
Draft diff: token overlap with cleanup
Trust decay: exponential, 30-day half-life
Cache: Durable Objects per conversation, 7-day TTL
Context strategy: minimal live context, retrieval-first, structured data in DB, everything else behind search

System Boundary (High Level)

Support Platform (Turborepo)
- apps/web: Dashboard
- apps/slack: Slack approvals bot
- apps/front: Front plugin
- packages/core: agent, tools, workflows, registry
- packages/sdk: integration contract + adapters
- packages/cli: skill CLI

External:
- Front (source of truth for conversations)
- Stripe Connect (refunds)
- Slack (HITL approvals)
- Upstash Vector (hybrid retrieval)
- Axiom + Langfuse (observability)

Implementation Index (PRD Format)

Phases are PR-ready units. Use the phase docs to execute.

Phase 0: Ops Readiness (No-Code Setup)
- ./phases/01-ops.md
Phase 0.5: Bedrock Repo
- ./phases/02-bedrock.md
Phase 1: Registry + Ingestion
- ./phases/03-registry-ingestion.md
Phase 2: Agent Core + Actions
- ./phases/04-agent-actions.md
Phase 3: HITL Surfaces
- ./phases/05-hitl-surfaces.md
Phase 4: SDK + First App
- ./phases/06-sdk-first-app.md
Phase 5: Stripe Connect
- ./phases/07-stripe-connect.md
Phase 6: Vector + Trust + Auto-send
- ./phases/08-vector-trust.md
Phase 7: Polish + Ops
- ./phases/09-polish-ops.md

Reference Index

Use these for detailed implementation and policies.

Architecture + Boundaries: ./reference/60-architecture.md
Tech Stack + Deploy Targets: ./reference/61-stack-runtime.md
Event Ingestion: ./reference/62-event-ingestion.md
Webhook Signing: ./reference/63-webhook-signing.md
Agent + Tools: ./reference/64-agent-tools.md
Workflows (Inngest): ./reference/65-workflows.md
HITL (Slack/Front/Dashboard): ./reference/66-hitl.md
SDK + Adapter: ./reference/67-sdk.md
CLI (skill): ./reference/68-cli.md
Data Model: ./reference/69-data-model.md
Observability: ./reference/70-observability.md
Vector Search: ./reference/71-vector-search.md
Agent Context Strategy: ./reference/72-context-strategy.md
Secrets + Encryption: ./reference/73-secrets.md
Defaults (Retention, SLAs, Policies): ./reference/74-defaults.md
Distributed Systems Patterns: ./reference/75-distributed-patterns.md

Raw

01-ops.md

Phase 0 - Ops Readiness (No-Code Setup)

Goal

All external accounts, keys, and access are ready so coding can proceed without pauses.

Required Runbook (Step-by-Step)

1) Front

Create a Front app with Webhooks + API access
Generate webhook signing secret
Create inboxes (one per product) + routing rules
Create an API token with scopes for conversations, messages, tags, and drafts
Capture: FRONT_API_TOKEN, FRONT_WEBHOOK_SECRET, inbox IDs per product

2) Slack

Create Slack app + bot user
Enable Interactivity & Shortcuts
Enable Events API (as needed)
Install app to workspace
Create approval channel(s)
Capture: SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET, approval channel ID

3) Stripe Connect

Confirm platform account access
Create Connect OAuth app
Create webhook endpoint for Connect events
Enable test mode
Capture: STRIPE_SECRET_KEY, STRIPE_CONNECT_CLIENT_ID, STRIPE_CONNECT_CLIENT_SECRET, STRIPE_CONNECT_WEBHOOK_SECRET

4) Upstash Vector

Create Vector DB (hybrid search defaults)
Capture: UPSTASH_VECTOR_URL, UPSTASH_VECTOR_TOKEN

5) Axiom + Langfuse

Axiom: create dataset + API token
Langfuse: create project + keys
Capture: AXIOM_DATASET, AXIOM_TOKEN, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY

6) BetterAuth

Create BetterAuth app
Configure session cookie name + domain
Configure OAuth providers if needed
Capture: BETTERAUTH_SECRET + provider credentials

7) PlanetScale

Create database + branch (prod + dev)
Create least-privilege user
Capture: DATABASE_URL

8) Cloudflare Workers / DO

Create Cloudflare account + Workers project
Create Durable Objects namespace
Capture: CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKEN, DO namespace ID

9) Vercel

Create Vercel projects (web, slack, front plugin)
Set env var placeholders for all services
Capture: project IDs for CI config

10) Secrets Storage

Create KMS key for envelope encryption
Ensure app_secrets table exists (see schema)
Capture: KMS_KEY_ID

PR-Ready Checklist

All credentials stored in secret manager/ENV
Validation: can hit Front/Slack/Stripe/Upstash with a test request

Validation / Tests

Smoke test each provider with minimal API call

Raw

02-bedrock.md

Phase 0.5 - Bedrock Repo

Goal

Greenfield repo with CI, lint, and base modules.

Deliverables

Turborepo scaffold (apps/web, apps/slack, apps/front, packages/core, packages/sdk, packages/cli)
Tooling wired (typescript-go, oxlint, biome)
Base CI + lint

PR-Ready Checklist

Repo boots (pnpm dev for web, minimal hello routes)
Lint and typecheck pass

Validation / Tests

Unit test harness bootstrapped
CI pipeline passes

Raw

03-registry-ingestion.md

Phase 1 - Registry + Ingestion

Goal

Deliverables

App registry + DB schema
Front webhook ingestion route + signature verify
DO cache (metadata + last 10 messages + last draft)
Inngest event dispatch wired
Retrieval-first guardrail: ingestion stores minimal context + search hooks
Retrieval-first guardrail: ingestion stores minimal context + search hooks

PR-Ready Checklist

skill init registers app and stores webhook secret
HMAC verification passes + rejects replays

Validation / Tests

Unit: signature verify, replay protection
Integration: webhook → inngest event

Raw

04-agent-actions.md

Phase 2 - Agent Core + Actions

Goal

Agent with action tools + audited execution.

Deliverables

Mastra support-agent
Tool registry and action executor
Audit log entries for actions
Retrieval-first prompt assembly (top-k snippets + summaries only)
Retrieval-first prompt assembly (top-k snippets + summaries only)

PR-Ready Checklist

Agent produces action + draft on a fixture
Tool execution writes audit record

Validation / Tests

Unit: tool requiresApproval logic, action logging
Integration: agent → tool exec → audit

Raw

05-hitl-surfaces.md

Phase 3 - HITL Surfaces

Goal

Human approvals in Slack + Front plugin; draft diff scoring.

Deliverables

Slack approval workflow
Front plugin (read + action)
Draft diff scoring (token overlap + cleanup)

PR-Ready Checklist

Approve/reject updates action status
Draft scoring classifies unmodified/edited/rewritten

Validation / Tests

Integration: Slack approval → action update
Unit: tokenization + diff scoring

Raw

06-sdk-first-app.md

Phase 4 - SDK + First App

Goal

SDK + Next adapter + first app integration.

Deliverables

@skillrecordings/support-sdk
createSupportRoutes adapter
skill doctor/test commands
First app integration (TT)

PR-Ready Checklist

skill doctor passes on first app
SDK routes respond with signature verification

Validation / Tests

Integration: SDK routes end-to-end
E2E: draft → send → HITL score

Raw

07-stripe-connect.md

Phase 5 - Stripe Connect

Goal

Centralized refunds via Connect.

Deliverables

OAuth connect flow
Refund processing + revokeAccess
Idempotency keys for Stripe actions

PR-Ready Checklist

OAuth completes and stores account
Refund flow logs audit + revokes access

Validation / Tests

Integration: Stripe webhook ingestion
E2E: refund in test mode

Raw

08-vector-trust.md

Phase 6 - Vector + Trust + Auto-send

Goal

Hybrid retrieval and trust-based auto-send gating.

Deliverables

Upstash Vector single index with type filters
Retrieval flow in agent context
Trust scoring + auto-send gating
Retrieval-first context pipeline (hybrid + keyword fallback)
Retrieval-first context pipeline (hybrid + keyword fallback)

PR-Ready Checklist

Retrieval returns similar tickets/knowledge/response examples
Auto-send gated by trust + confidence

Validation / Tests

Integration: upsert/query with includeData
E2E: trust thresholds block/allow auto-send

Raw

09-polish-ops.md

Phase 7 - Polish + Ops

Goal

Observability, retention, stability.

Deliverables

Axiom tracing + Langfuse
Retention policies enforced
Rate limit and backpressure

PR-Ready Checklist

Traces emitted on webhook → agent → action
Dead-letter + alerts wired

Validation / Tests

E2E: multi-app routing
E2E: rate limit + cache behavior

Raw

60-architecture.md

Architecture + Boundaries

System Boundary

Support Platform (Turborepo)
- apps/web: Dashboard
- apps/slack: Slack approvals bot
- apps/front: Front plugin
- packages/core: agent, tools, workflows, registry
- packages/sdk: integration contract + adapters
- packages/cli: skill CLI

External:
- Front (source of truth for conversations)
- Stripe Connect (refunds)
- Slack (HITL approvals)
- Upstash Vector (hybrid retrieval)
- Axiom + Langfuse (observability)

Vision

Success Criteria

Reduce human touches by 80%
Sub-minute response time (draft within 60s)
Full traceability (every decision/action/approval logged)
Multi-app robustness (adding a new app is a skill init away)

Raw

61-stack-runtime.md

Tech Stack + Deploy Targets

Tech Stack

Layer	Technology	Rationale
Monorepo	Turborepo + Bun	Fast, modern, good DX
Build	typescript-go	10x faster type checking
Lint	oxlint + oxlint-tsgolint + biome	Fast linting + TS diagnostics + formatting
Framework	Next.js + Turbopack	RSC, streaming, Vercel hosting
Agent	Mastra + AI SDK	TypeScript-native, good tooling story
Workflows	Inngest	Durable execution, retries, scheduling
State	Cloudflare Durable Objects	Real-time, low-latency conversation state
Database	PlanetScale	MySQL at edge
Vector DB	Upstash Vector	Serverless, hybrid search, simple API
Observability	Axiom + Langfuse	Logs/traces + LLM-specific observability
Auth	BetterAuth	App registration + team access

Deploy Targets

Component	Runtime
Webhook ingestion	Cloudflare Workers
Conversation cache	Durable Objects
Vector search	Upstash Vector
Workflows	Inngest + Vercel
Dashboard + CLI API + Front plugin	Next.js on Vercel
Slack bot	Vercel

Raw

62-event-ingestion.md

Event Ingestion

Source: Front webhooks, Stripe webhooks (via Connect), app-reported events

export async function handleFrontWebhook(req: Request) {
  const verified = await verifySupportSignature(req)
  if (!verified) return new Response('Unauthorized', { status: 401 })

  const event = await parseFrontEvent(req)
  const app = await appRegistry.findByFrontInbox(event.inboxId)
  if (!app) return new Response('Unknown inbox', { status: 404 })

  await inngest.send({
    name: 'front/event.received',
    data: { event, appId: app.id }
  })

  return new Response('OK', { status: 200 })
}

Front events:

inbound_received → classify, extract intent, propose response/action
outbound_sent → log, update state, detect promises made
assignee_changed → handoff context
tag_added → trigger workflows (urgent → Slack escalation)

Raw

63-webhook-signing.md

Webhook Signing

HMAC-SHA256 over raw body with shared secret. Stripe-style format with replay protection.

x-support-signature: t=1705512000,v1=5257a869...,v1=oldkeysig...

Timestamp (t) + 5 minute tolerance
Multiple v1 signatures for key rotation
Verify: HMAC-SHA256(timestamp + "." + rawBody, webhookSecret)

function verifySignature(payload: string, header: string, secrets: string[]): boolean {
  const { timestamp, signatures } = parseHeader(header)

  if (Date.now() - timestamp > 5 * 60 * 1000) return false

  const signedPayload = `${timestamp}.${payload}`

  return secrets.some(secret =>
    signatures.some(sig =>
      timingSafeEqual(hmacSha256(signedPayload, secret), Buffer.from(sig, 'hex'))
    )
  )
}

Raw

64-agent-tools.md

Agent + Tools

Agent Core (Mastra)

import { Agent } from '@mastra/core'
import { supportTools } from './tools'

export const supportAgent = new Agent({
  name: 'support-agent',
  instructions: `
    You are a support agent for Skill Recordings products.

    Your goal is to resolve customer issues quickly and empathetically.

    ## Authority Levels

    AUTO-APPROVE (do immediately):
    - Magic link requests
    - Password reset requests
    - Refunds within 30 days of purchase
    - Transfers within 14 days of purchase
    - Email/name updates

    REQUIRE-APPROVAL (draft action, wait for human):
    - Refunds 30-45 days after purchase
    - Transfers after 14 days
    - Bulk seat management
    - Account deletions

    ALWAYS-ESCALATE (flag for human, do not act):
    - Angry/frustrated customers (detect sentiment)
    - Legal language (lawsuit, lawyer, etc.)
    - Repeated failed interactions
    - Anything you're uncertain about

    ## Response Style
    - Be personal, use first names
    - Be concise, not verbose
    - Don't apologize excessively
    - Focus on resolution, not explanation
  `,
  model: { provider: 'anthropic', name: 'claude-sonnet-4-20250514' },
  tools: supportTools,
})

Tools (Actions)

import { createTool } from '@mastra/core'

export const supportTools = {
  lookupUser: createTool({
    name: 'lookup_user',
    description: 'Look up a user by email to get their account details and purchase history',
    parameters: z.object({
      email: z.string().email(),
      appId: z.string(),
    }),
    execute: async ({ email, appId }) => {
      const app = await appRegistry.get(appId)
      return app.integration.lookupUser(email)
    },
  }),

  processRefund: createTool({
    name: 'process_refund',
    description: 'Process a refund for a purchase. Use only within policy.',
    parameters: z.object({
      purchaseId: z.string(),
      appId: z.string(),
      reason: z.string(),
    }),
    requiresApproval: (params, context) => {
      const purchase = context.purchases.find(p => p.id === params.purchaseId)
      const daysSincePurchase = daysBetween(purchase.purchasedAt, new Date())
      return daysSincePurchase > 30
    },
    execute: async ({ purchaseId, appId, reason }, { approvalId }) => {
      const stripeRefund = await stripe.refunds.create({
        charge: purchase.stripeChargeId,
      }, {
        stripeAccount: app.stripeAccountId,
      })

      await app.integration.revokeAccess({
        purchaseId,
        reason,
        refundId: stripeRefund.id,
      })

      await auditLog.record({
        action: 'refund',
        purchaseId,
        appId,
        approvalId,
        stripeRefundId: stripeRefund.id,
      })

      return { success: true, refundId: stripeRefund.id }
    },
  }),

  generateMagicLink: createTool({
    name: 'generate_magic_link',
    description: 'Generate a magic login link for a user',
    parameters: z.object({
      email: z.string().email(),
      appId: z.string(),
    }),
    execute: async ({ email, appId }) => {
      const app = await appRegistry.get(appId)
      return app.integration.generateMagicLink({ email, expiresIn: 300 })
    },
  }),

  transferPurchase: createTool({
    name: 'transfer_purchase',
    description: 'Transfer a purchase from one user to another',
    parameters: z.object({
      purchaseId: z.string(),
      fromEmail: z.string().email(),
      toEmail: z.string().email(),
      appId: z.string(),
    }),
    requiresApproval: (params, context) => {
      const purchase = context.purchases.find(p => p.id === params.purchaseId)
      const daysSincePurchase = daysBetween(purchase.purchasedAt, new Date())
      return daysSincePurchase > 14
    },
    execute: async ({ purchaseId, fromEmail, toEmail, appId }) => {
      const app = await appRegistry.get(appId)
      return app.integration.transferPurchase({
        purchaseId,
        fromUserId: context.user.id,
        toEmail,
      })
    },
  }),

  draftResponse: createTool({
    name: 'draft_response',
    description: 'Draft a response to send to the customer via Front',
    parameters: z.object({
      conversationId: z.string(),
      body: z.string(),
      appId: z.string(),
    }),
    execute: async ({ conversationId, body, appId }) => {
      await front.conversations.createDraft(conversationId, {
        body,
        author_id: 'support-bot',
      })
      return { drafted: true }
    },
  }),

  escalateToHuman: createTool({
    name: 'escalate_to_human',
    description: 'Escalate this conversation to a human support agent',
    parameters: z.object({
      conversationId: z.string(),
      reason: z.string(),
      urgency: z.enum(['low', 'medium', 'high']),
    }),
    execute: async ({ conversationId, reason, urgency }) => {
      await front.conversations.addTag(conversationId, 'needs-human')

      await slack.postMessage({
        channel: SUPPORT_CHANNEL,
        text: `🚨 Escalation needed`,
        blocks: [
          {
            type: 'section',
            text: { type: 'mrkdwn', text: `*Reason:* ${reason}\n*Urgency:* ${urgency}` },
          },
          {
            type: 'actions',
            elements: [
              { type: 'button', text: { type: 'plain_text', text: 'Open in Front' }, url: frontConversationUrl },
            ],
          },
        ],
      })

      return { escalated: true }
    },
  }),
}

Raw

65-workflows.md

Workflows (Inngest)

import { inngest } from './client'

export const handleInboundMessage = inngest.createFunction(
  {
    id: 'handle-inbound-message',
    throttle: {
      key: 'event.data.conversationId',
      limit: 1,
      period: '10s',
    },
  },
  { event: 'front/inbound_received' },
  async ({ event, step }) => {
    const { conversationId, appId, senderEmail } = event.data

    const context = await step.run('gather-context', async () => {
      const [user, messages, app] = await Promise.all([
        appRegistry.get(appId).integration.lookupUser(senderEmail),
        front.conversations.listMessages(conversationId),
        appRegistry.get(appId),
      ])
      return { user, messages, app }
    })

    const agentResult = await step.run('agent-reasoning', async () => {
      return supportAgent.run({
        messages: [
          {
            role: 'user',
            content: `
              New support message received.

              Customer: ${context.user?.name || senderEmail}
              Email: ${senderEmail}
              Product: ${context.app.name}

              Purchase history:
              ${JSON.stringify(context.user?.purchases || [], null, 2)}

              Conversation:
              ${context.messages.map(m => `${m.author}: ${m.text}`).join('\n')}

              Determine the intent and propose an action.
            `,
          },
        ],
        context,
      })
    })

    if (agentResult.action?.requiresApproval) {
      await step.run('request-approval', async () => {
        await requestApproval({
          action: agentResult.action,
          conversationId,
          appId,
          agentReasoning: agentResult.reasoning,
        })
      })
    } else if (agentResult.action) {
      await step.run('execute-action', async () => {
        await executeAction(agentResult.action)
      })
    }

    if (agentResult.draftResponse) {
      await step.run('create-draft', async () => {
        await front.conversations.createDraft(conversationId, {
          body: agentResult.draftResponse,
        })
      })
    }

    return { processed: true, action: agentResult.action }
  }
)

Raw

66-hitl.md

HITL (Slack, Front, Dashboard)

Slack Approval Flow

import { App } from '@slack/bolt'

app.action('approve_action', async ({ ack, body, client }) => {
  await ack()

  const { actionId, approverId } = JSON.parse(body.actions[0].value)
  const result = await executeApprovedAction(actionId, approverId)

  await client.chat.update({
    channel: body.channel.id,
    ts: body.message.ts,
    text: `✅ Approved by <@${approverId}>`,
    blocks: [
      {
        type: 'section',
        text: {
          type: 'mrkdwn',
          text: `✅ *Approved* by <@${approverId}>\n\nResult: ${JSON.stringify(result)}`
        },
      },
    ],
  })
})

app.action('reject_action', async ({ ack, body, client }) => {
  await ack()

  await client.views.open({
    trigger_id: body.trigger_id,
    view: {
      type: 'modal',
      callback_id: 'rejection_reason',
      title: { type: 'plain_text', text: 'Rejection Reason' },
      blocks: [
        {
          type: 'input',
          block_id: 'reason',
          element: { type: 'plain_text_input', action_id: 'reason_input' },
          label: { type: 'plain_text', text: 'Why are you rejecting this action?' },
        },
      ],
      submit: { type: 'plain_text', text: 'Submit' },
      private_metadata: JSON.stringify({ actionId: body.actions[0].value }),
    },
  })
})

Approval Surfaces

Slack: real-time push, approve/reject, modal for rejection reason
Dashboard: queue view, bulk actions, search, audit trail
Front plugin: read + action

Front Plugin Actions

Read:

Customer context
Purchases
Conversation summary
Trust scores

Action:

Approve/edit/reject draft
Quick refund
Quick magic link
Escalate

Draft Comparison (HITL Scoring)

Token overlap heuristic with pre-tokenize cleanup.

function tokenize(text: string): string[] {
  const withoutSig = text
    .replace(/^--\s*[\s\S]*$/m, '')
    .replace(/^(Best|Thanks|Cheers|Regards),?\s*[\s\S]*$/mi, '')

  const withoutQuotes = withoutSig.replace(/^>.*$/gm, '')

  return withoutQuotes
    .toLowerCase()
    .split(/\W+/)
    .filter(t => t.length > 2)
}

function diffScore(draft: string, sent: string): 'unmodified' | 'edited' | 'rewritten' {
  const draftTokens = new Set(tokenize(draft))
  const sentTokens = new Set(tokenize(sent))
  const intersection = [...draftTokens].filter(t => sentTokens.has(t))
  const overlap = intersection.length / Math.max(draftTokens.size, sentTokens.size)

  if (overlap > 0.95) return 'unmodified'
  if (overlap > 0.50) return 'edited'
  return 'rewritten'
}

Raw

67-sdk.md

SDK + Adapter

SupportIntegration Interface

export interface SupportIntegration {
  lookupUser(email: string): Promise<User | null>
  getPurchases(userId: string): Promise<Purchase[]>
  getSubscriptions?(userId: string): Promise<Subscription[]>

  revokeAccess(params: {
    purchaseId: string
    reason: string
    refundId: string
  }): Promise<ActionResult>

  transferPurchase(params: {
    purchaseId: string
    fromUserId: string
    toEmail: string
  }): Promise<ActionResult>

  generateMagicLink(params: {
    email: string
    expiresIn: number
  }): Promise<{ url: string }>

  updateEmail?(params: {
    userId: string
    newEmail: string
  }): Promise<ActionResult>

  updateName?(params: {
    userId: string
    newName: string
  }): Promise<ActionResult>

  getClaimedSeats?(bulkCouponId: string): Promise<ClaimedSeat[]>
}

Next.js Adapter

import { createSupportHandler } from './handler'
import type { SupportIntegration } from '../types'

export function createSupportRoutes(
  integration: SupportIntegration,
  options: { webhookSecret: string }
) {
  const handler = createSupportHandler(integration, options)

  return {
    GET: handler,
    POST: handler,
  }
}

Raw

68-cli.md

CLI (skill)

The CLI is the primary agentic interface. Claude Code and OpenCode agents use it to investigate and act.

Operator Commands

skill lookup joel@example.com --app tt
skill purchases joel@example.com --app tt
skill conversation cnv_abc123
skill history joel@example.com --app tt

skill refund ch_xxx --app tt --reason "requested"
skill transfer pur_xxx --to new@email.com --app tt
skill magic-link joel@example.com --app tt
skill cancel-subscription sub_xxx --app tt

Agent Mode

skill agent --context cnv_abc123
skill agent --json "refund the last charge for joel@example.com on total-typescript"
echo '{"task": "investigate access issue", "email": "joel@example.com"}' | skill agent --json

Agent mode output (JSON):

{
  "reasoning": "Customer purchased 15 days ago, within refund window...",
  "action": {
    "type": "refund",
    "purchaseId": "pur_abc",
    "amount": 29900
  },
  "result": {
    "success": true,
    "refundId": "re_xyz"
  },
  "draftResponse": "Hi Joel, I've processed your refund..."
}

Bulk Ops

skill refund --bulk --csv refunds.csv --dry-run
skill transfer --bulk --csv transfers.csv

Observability

skill logs --app tt --since 1h
skill trace tr_xxx
skill metrics --app tt --period 7d

Eval & Training Data

skill export conversations --since 2023-01-01 --app tt --format jsonl > tt.jsonl
skill enrich tt.jsonl --stripe-outcomes --resolution-time > tt_enriched.jsonl
skill sample tt_enriched.jsonl --strategy stratified --size 500 > tt_eval.jsonl

skill eval run --dataset tt_eval.jsonl --agent support-agent
skill eval report --format markdown

Raw

69-data-model.md

Data Model

CREATE TABLE apps (
  id TEXT PRIMARY KEY,
  slug TEXT UNIQUE NOT NULL,
  name TEXT NOT NULL,

  front_inbox_id TEXT NOT NULL,

  stripe_account_id TEXT,
  stripe_connected BOOLEAN DEFAULT FALSE,

  integration_base_url TEXT NOT NULL,
  webhook_secret TEXT NOT NULL,
  capabilities TEXT[] NOT NULL,

  auto_approve_refund_days INTEGER DEFAULT 30,
  auto_approve_transfer_days INTEGER DEFAULT 14,
  escalation_slack_channel TEXT,

  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE conversations (
  id TEXT PRIMARY KEY,
  front_conversation_id TEXT UNIQUE NOT NULL,
  app_id TEXT REFERENCES apps(id),

  customer_email TEXT NOT NULL,
  customer_name TEXT,

  status TEXT DEFAULT 'open',
  assigned_to TEXT,

  last_agent_run_id TEXT,
  last_agent_action TEXT,

  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE actions (
  id TEXT PRIMARY KEY,
  conversation_id TEXT REFERENCES conversations(id),
  app_id TEXT REFERENCES apps(id),

  type TEXT NOT NULL,
  parameters JSONB NOT NULL,

  requires_approval BOOLEAN DEFAULT FALSE,
  approved_by TEXT,
  approved_at TIMESTAMP,
  rejected_by TEXT,
  rejected_at TIMESTAMP,
  rejection_reason TEXT,

  executed_at TIMESTAMP,
  result JSONB,
  error TEXT,

  trace_id TEXT,
  langfuse_trace_id TEXT,

  created_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE approval_requests (
  id TEXT PRIMARY KEY,
  action_id TEXT REFERENCES actions(id),

  slack_message_ts TEXT,
  slack_channel TEXT,

  status TEXT DEFAULT 'pending',

  agent_reasoning TEXT,

  expires_at TIMESTAMP,
  created_at TIMESTAMP DEFAULT NOW()
);

Raw

70-observability.md

Observability

Tracing (Axiom)

import { trace } from '@axiomhq/js'

export function withTracing<T>(
  name: string,
  fn: () => Promise<T>,
  attributes?: Record<string, string>
): Promise<T> {
  return trace(name, async (span) => {
    span.setAttributes(attributes)
    try {
      const result = await fn()
      span.setStatus({ code: 'OK' })
      return result
    } catch (error) {
      span.setStatus({ code: 'ERROR', message: error.message })
      throw error
    }
  })
}

LLM Observability (Langfuse)

import { Langfuse } from 'langfuse'

const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  secretKey: process.env.LANGFUSE_SECRET_KEY,
})

export async function traceAgentRun(agentRun: AgentRun, context: ConversationContext) {
  const trace = langfuse.trace({
    name: 'support-agent',
    metadata: {
      conversationId: context.conversationId,
      appId: context.appId,
      userEmail: context.userEmail,
    },
  })

  const generation = trace.generation({
    name: 'agent-reasoning',
    model: 'claude-sonnet-4-20250514',
    input: agentRun.input,
    output: agentRun.output,
    usage: agentRun.usage,
  })

  return { traceId: trace.id, generationId: generation.id }
}

Raw

71-vector-search.md

Vector Search (Upstash Vector)

Hybrid search (dense + sparse) with hosted embeddings. Single index, type filters.

const index = new Index({
  url: process.env.UPSTASH_VECTOR_URL,
  token: process.env.UPSTASH_VECTOR_TOKEN,
})

await index.upsert([{
  id: conversationId,
  data: redactPII(messageText),
  metadata: { type: 'conversation', appId, category, resolution }
}])

const results = await index.query({
  data: redactPII(queryText),
  topK: 5,
  filter: `appId = '${appId}'`,
  includeData: true,
  includeMetadata: true,
})

PII Redaction

function redactPII(text: string, knownNames: string[] = []): string {
  let redacted = text
    .replace(/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g, '[EMAIL]')
    .replace(/(\+?1[-.\s]?)?\(?[0-9]{3}\)?[-.\s]?[0-9]{3}[-.\s]?[0-9]{4}/g, '[PHONE]')
    .replace(/\b(?:\d{4}[-\s]?){3}\d{4}\b/g, '[CARD]')

  if (knownNames.length > 0) {
    redacted = redacted.replace(
      new RegExp(knownNames.map(escapeRegex).join('|'), 'gi'),
      '[NAME]'
    )
  }

  return redacted
}

Single Index with Type Filter

interface VectorDocument {
  id: string
  data: string
  metadata: {
    type: 'conversation' | 'knowledge' | 'response'
    appId: string
    category?: MessageCategory
    resolution?: 'refund' | 'transfer' | 'info' | 'escalated'
    customerSentiment?: 'positive' | 'neutral' | 'negative'
    touchCount?: number
    resolvedAt?: string
    source?: 'docs' | 'faq' | 'policy' | 'canned-response'
    title?: string
    lastUpdated?: string
    trustScore?: number
    usageCount?: number
    conversationId?: string
  }
}

Agent Retrieval Flow

async function buildAgentContext(message: string, appId: string) {
  const query = redactPII(message)

  const [similarTickets, knowledge, goodResponses] = await Promise.all([
    index.query({
      data: query,
      topK: 3,
      filter: `appId = '${appId}' AND type = 'conversation' AND resolution != 'escalated'`,
      includeData: true,
      includeMetadata: true,
    }),
    index.query({
      data: query,
      topK: 5,
      filter: `appId = '${appId}' AND type = 'knowledge'`,
      includeData: true,
      includeMetadata: true,
    }),
    index.query({
      data: query,
      topK: 3,
      filter: `appId = '${appId}' AND type = 'response' AND trustScore > 0.85`,
      includeData: true,
      includeMetadata: true,
    }),
  ])

  return { similarTickets, knowledge, goodResponses }
}

Optional: Cohere Rerank

async function buildAgentContextWithRerank(message: string, appId: string) {
  const query = redactPII(message)

  const [denseResults, sparseResults] = await Promise.all([
    index.query({ data: query, queryMode: 'DENSE', topK: 30, filter: `appId = '${appId}'`, includeData: true }),
    index.query({ data: query, queryMode: 'SPARSE', topK: 30, filter: `appId = '${appId}'`, includeData: true }),
  ])

  const candidates = dedupeById([...denseResults, ...sparseResults])

  const reranked = await cohere.rerank({
    model: 'rerank-v4.0-pro',
    query: message,
    documents: candidates.map(c => c.data),
    topN: 10,
  })

  const results = reranked.results.map(r => candidates[r.index])
  return {
    similarTickets: results.filter(r => r.metadata.type === 'conversation').slice(0, 3),
    knowledge: results.filter(r => r.metadata.type === 'knowledge').slice(0, 5),
    goodResponses: results.filter(r => r.metadata.type === 'response').slice(0, 3),
  }
}

Raw

72-context-strategy.md

Agent Context Strategy

Based on Malte Ubl's guidance: keep live context tiny, store bulk data in files/DB, and rely on search.

Principles

Put only a small slice of data into live context (recent messages + minimal app state).
Organize larger context into files (phase docs + references) and fetch on demand.
Keep structured data in databases and access via tools.
Put everything else behind search (keyword + hybrid retrieval).

Guardrails (Retrieval-First)

Never assemble full conversation history in the prompt.
Always retrieve top-k snippets (hybrid + keyword fallback) and summarize.
Prefer structured tool calls for purchases/entitlements/approvals/trust stats.
Enforce a strict context budget; summarize or drop long tails.

Guardrails (Retrieval-First)

Never assemble full conversation history in the prompt.
Always retrieve top-k snippets (hybrid + keyword fallback) and summarize.
Prefer structured tool calls for purchases/entitlements/approvals/trust stats.
Enforce a strict context budget; summarize or drop long tails.

Applied Defaults

DO cache holds only metadata + last 10 message previews + last draft.
Raw message bodies stored briefly (30 days), then only hashes + IDs.
Retrieval pipeline: hybrid search with filters; top-k snippets only.
Agent uses tools to fetch context rather than manual paste.

Raw

73-secrets.md

Secrets + Encryption

Avoid platform env sprawl. Store third-party secrets encrypted in the DB.

Approach

Encrypt at rest with envelope encryption
Per-app context for crypto isolation
KMS-backed master key (AWS/GCP/Azure) for KEK

What Gets Encrypted

Stripe keys (Connect, webhook secrets)
Front/Slack tokens
OAuth refresh tokens
Any app integration secrets

const dek = await kms.generateDataKey({ keyId: MASTER_KEY_ID })
const ciphertext = encryptWithDek(plaintext, dek.plaintext)
store({ ciphertext, encryptedDek: dek.ciphertextBlob, appId })

Rotation

Support multiple KEKs (active + previous)
Re-encrypt DEKs asynchronously during low-traffic windows
Log decrypt usage for audit

Suggested Schema

CREATE TABLE app_secrets (
  id TEXT PRIMARY KEY,
  app_id TEXT REFERENCES apps(id),
  provider TEXT NOT NULL,
  ciphertext BLOB NOT NULL,
  encrypted_dek BLOB NOT NULL,
  key_version INTEGER NOT NULL,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

Raw

74-defaults.md

Defaults (Retention, SLAs, Policies)

Auto-send Trust

Trust threshold: 0.85
Minimum samples: 50
Confidence floor: 0.90
Never auto-send: angry-customer, legal, team-license, other

Retention

Actions/audit log: 18 months
Approval requests: 90 days
Conversation metadata: 24 months
Raw message bodies in platform DB: 30 days (Front remains source of truth)
DO cache: 7 days

SLA / Latency Targets

Webhook → draft: p95 < 60s, p99 < 120s
Approval → execution: p95 < 5m
Dashboard load: p95 < 2s

Error + Retry Policy

Front/Slack: exponential backoff, jitter, max 5 retries
Stripe: max 3 retries, idempotency keys required
Dead-letter after max retries; alert + manual retry

Event Schema Versioning

Versioned event envelope: version, type, source, occurredAt
Backward-compatible changes only; breaking changes require new type or version
Deprecate old versions after 90 days

PII Policy

Redact emails/phones/cards in vectors
Store message bodies only 30 days; keep hashes + IDs after
Avoid storing attachments; keep references/IDs only

Raw

75-distributed-patterns.md

Distributed Systems Patterns

Idempotency + dedupe: all actions accept an actionId and ignore repeats; Stripe actions use idempotency keys
Transactional outbox: persist inbound events + intended side effects in DB, then emit to Inngest
Cache staleness: DO cache is best-effort; expose lastSyncedAt, allow forced refresh, reconcile periodically
Backpressure: batch vector upserts/exports, cap concurrency, respect provider rate limits
Audit log = replay log: keep immutable action/event history for replays and debugging