Skip to content

Instantly share code, notes, and snippets.

@tomfuertes
Created February 25, 2026 16:53
Show Gist options
  • Select an option

  • Save tomfuertes/0f97fb30114f58f3de8e22caeb12e7c0 to your computer and use it in GitHub Desktop.

Select an option

Save tomfuertes/0f97fb30114f58f3de8e22caeb12e7c0 to your computer and use it in GitHub Desktop.
Claude Code .claude/skills examples - UAT testing an AI agent API (Ghostfolio)

.claude/skills examples - Ghostfolio AI Agent

Project-local Claude Code skills for UAT testing an AI agent endpoint built on top of Ghostfolio.

What are Claude Code skills?

Skills (.claude/skills/<name>/SKILL.md) are slash commands that expand into prompt instructions when you type /<name> in Claude Code. They can:

  • Run bash commands with \! prefix in the frontmatter to inject live context
  • Gate tool usage via allowed-tools
  • Skip a model round-trip with disable-model-invocation: true (inline expansion only)

File structure

.claude/skills/
  uat/
    SKILL.md                         ← /uat <message> - hit the agent API
    scripts/
      setup.sh                       ← creates test user, imports sample data
      create-test-accounts.sh        ← pre-creates a pool of test accounts
  uat-verify/
    SKILL.md                         ← /uat-verify activities|portfolio|holdings

Skills

/uat <message>

Send a natural-language message to the agent API (POST /api/v1/agent/chat), auto-manage JWT auth, display tool calls + response. Multi-turn aware via conversationId.

/uat-verify <view>

Use agent-browser to log into the Ghostfolio Angular UI, navigate to the target page, screenshot it, and analyze what's visible. Confirms the agent's mutations actually show up in the UI.

Dependencies

  • agent-browser installed globally (npm install -g agent-browser)
  • jq available on PATH
  • Dev server running (npm run dev - API on :3333, client on :4200)
name description argument-hint disable-model-invocation allowed-tools
uat
Send a message to the Ghostfolio agent API and display the response. Use for testing agent tools end-to-end against a running dev server.
<message to send to agent>
true
Bash, Read, Write

UAT - Agent API Tester

Send $ARGUMENTS to the Ghostfolio agent at http://localhost:3333/api/v1/agent/chat and display the response.

Steps

  1. Check for credentials - Read .uat-creds.json in the repo root. If it doesn't exist, run setup first.

  2. Setup (if no creds) - Run .claude/skills/uat/scripts/setup.sh to create a test user and import sample data.

  3. Load credentials - Read .uat-creds.json:

    { "accessToken": "...", "authToken": "..." }

    authToken is the JWT Bearer token. accessToken is the anonymous auth token for refreshing.

  4. Send the message - POST to the agent:

    curl -s -X POST http://localhost:3333/api/v1/agent/chat \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer <authToken>" \
      -d '{"message": "$ARGUMENTS"}'
  5. Handle auth errors - If you get a 401, refresh the JWT:

    curl -s -X POST http://localhost:3333/api/v1/auth/anonymous \
      -H "Content-Type: application/json" \
      -d '{"accessToken": "<accessToken from creds>"}'

    Save the new authToken back to .uat-creds.json, then retry the agent call.

  6. Display results - Format and show:

    • The response text from the agent
    • Any toolCalls array (tool name + inputs + output for each)
    • The conversationId (save it to .uat-creds.json as conversationId for multi-turn continuity)
    • Any errors with status code and body
  7. Multi-turn - If .uat-creds.json has a conversationId, include it in the request body:

    { "message": "$ARGUMENTS", "conversationId": "..." }

    To start a fresh conversation, delete conversationId from .uat-creds.json.

Credentials file format

.uat-creds.json (gitignored):

{
  "accessToken": "anonymous-access-token",
  "authToken": "jwt-bearer-token",
  "conversationId": "optional-for-multi-turn"
}

Troubleshooting

  • Connection refused: API isn't running. Run npm run dev:api (or npm run dev for deps + api).
  • 401 Unauthorized: Refresh JWT via /auth/anonymous (step 5 above).
  • 500 errors: Check API logs. Common cause: LLM config missing from DB (OpenRouter key not set).
  • No .uat-creds.json: Run setup script manually: bash .claude/skills/uat/scripts/setup.sh
#!/usr/bin/env bash
# Creates 10 anonymous test accounts against a running Ghostfolio API.
# Usage: bash .claude/skills/uat/scripts/create-test-accounts.sh [dev|prod]
set -euo pipefail
ENV="${1:-dev}"
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
OUTFILE="$REPO_ROOT/test-accounts-${ENV}.json"
COUNT=10
case "$ENV" in
dev) API="http://localhost:3333/api/v1" ;;
prod) API="https://your-app.up.railway.app/api/v1" ;;
*) echo "Usage: $0 [dev|prod]"; exit 1 ;;
esac
echo "Creating $COUNT test accounts against $API"
echo "Output: $OUTFILE"
echo ""
# Check API is reachable
if ! curl -sf "$API/health" > /dev/null 2>&1; then
echo "ERROR: API not reachable at $API"
case "$ENV" in
dev) echo "Start it with: npm run dev or npm run docker:up" ;;
prod) echo "Check Railway dashboard" ;;
esac
exit 1
fi
echo "[" > "$OUTFILE"
for i in $(seq 1 "$COUNT"); do
RESP=$(curl -s -X POST "$API/user" -H "Content-Type: application/json")
ACCESS=$(echo "$RESP" | jq -r '.accessToken')
AUTH=$(echo "$RESP" | jq -r '.authToken')
if [ -z "$ACCESS" ] || [ "$ACCESS" = "null" ]; then
echo "ERROR: Failed to create account $i"
echo "Response: $RESP"
exit 1
fi
COMMA=","
[ "$i" -eq "$COUNT" ] && COMMA=""
echo " { \"account\": $i, \"accessToken\": \"$ACCESS\", \"authToken\": \"$AUTH\" }$COMMA" >> "$OUTFILE"
echo " [$i/$COUNT] Created"
done
echo "]" >> "$OUTFILE"
echo ""
echo "Done. $COUNT accounts saved to $OUTFILE"
#!/usr/bin/env bash
# UAT setup: creates a test user and imports sample portfolio data.
# Saves credentials to .uat-creds.json in the repo root.
# Run once before using /uat skill. Safe to re-run (overwrites creds).
set -euo pipefail
API="http://localhost:3333/api/v1"
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
CREDS_FILE="$REPO_ROOT/.uat-creds.json"
SAMPLE_FILE="$REPO_ROOT/test/import/ok/sample.json"
echo "UAT Setup - Ghostfolio Agent"
echo "=============================="
# 1. Check API is running
echo "[1/4] Checking API health..."
if ! curl -sf "$API/health" > /dev/null 2>&1; then
echo "ERROR: API not reachable at $API"
echo "Start it with: npm run dev:api"
echo "Or full stack: npm run dev"
exit 1
fi
echo " API is up."
# 2. Get test user credentials
echo "[2/4] Getting test user credentials..."
TEST_ACCOUNTS="$REPO_ROOT/test-accounts.json"
if [ -f "$TEST_ACCOUNTS" ]; then
# Use first unused account from pre-created pool
ACCOUNT_COUNT=$(jq length "$TEST_ACCOUNTS")
# Track which account we're using via .uat-account-index
INDEX_FILE="$REPO_ROOT/.uat-account-index"
if [ -f "$INDEX_FILE" ]; then
ACCOUNT_INDEX=$(cat "$INDEX_FILE")
else
ACCOUNT_INDEX=0
fi
if [ "$ACCOUNT_INDEX" -ge "$ACCOUNT_COUNT" ]; then
ACCOUNT_INDEX=0
echo " Wrapped around to first account."
fi
ACCESS_TOKEN=$(jq -r ".[$ACCOUNT_INDEX].accessToken" "$TEST_ACCOUNTS")
echo " Using pre-created account $ACCOUNT_INDEX from test-accounts.json"
# Get fresh JWT via login
LOGIN_RESPONSE=$(curl -s -X POST "$API/auth/anonymous" \
-H "Content-Type: application/json" \
-d "{\"accessToken\": \"$ACCESS_TOKEN\"}")
AUTH_TOKEN=$(echo "$LOGIN_RESPONSE" | jq -r '.authToken')
if [ -z "$AUTH_TOKEN" ] || [ "$AUTH_TOKEN" = "null" ]; then
echo "ERROR: Failed to login with account $ACCOUNT_INDEX"
exit 1
fi
# Increment index for next setup run
echo $(( ACCOUNT_INDEX + 1 )) > "$INDEX_FILE"
echo " Logged in. Next run will use account $(( ACCOUNT_INDEX + 1 ))."
else
# Fallback: create a new account
echo " No test-accounts.json found, creating new user..."
CREATE_RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$API/user" \
-H "Content-Type: application/json")
HTTP_CODE=$(echo "$CREATE_RESPONSE" | tail -1)
BODY=$(echo "$CREATE_RESPONSE" | head -n -1)
if [ "$HTTP_CODE" != "201" ] && [ "$HTTP_CODE" != "200" ]; then
echo "ERROR: Failed to create user (HTTP $HTTP_CODE)"
echo "Response: $BODY"
exit 1
fi
ACCESS_TOKEN=$(echo "$BODY" | jq -r '.accessToken')
AUTH_TOKEN=$(echo "$BODY" | jq -r '.authToken')
if [ -z "$ACCESS_TOKEN" ] || [ "$ACCESS_TOKEN" = "null" ]; then
echo "ERROR: No accessToken in response"
echo "Response: $BODY"
exit 1
fi
echo " User created. Access token: ${ACCESS_TOKEN:0:20}..."
fi
# 3. Save credentials
echo "[3/4] Saving credentials to .uat-creds.json..."
cat > "$CREDS_FILE" <<EOF
{
"accessToken": "$ACCESS_TOKEN",
"authToken": "$AUTH_TOKEN"
}
EOF
echo " Saved to: $CREDS_FILE"
# 4. Import sample portfolio
echo "[4/4] Importing sample portfolio..."
if [ ! -f "$SAMPLE_FILE" ]; then
echo " WARNING: Sample file not found at $SAMPLE_FILE - skipping import"
else
IMPORT_RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$API/import" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d @"$SAMPLE_FILE")
IMPORT_CODE=$(echo "$IMPORT_RESPONSE" | tail -1)
IMPORT_BODY=$(echo "$IMPORT_RESPONSE" | head -n -1)
if [ "$IMPORT_CODE" = "201" ] || [ "$IMPORT_CODE" = "200" ]; then
echo " Sample portfolio imported successfully."
else
echo " WARNING: Import returned HTTP $IMPORT_CODE"
echo " Response: $IMPORT_BODY"
echo " (Non-fatal - you can still test search_symbol and get_portfolio_summary)"
fi
fi
echo ""
echo "Setup complete!"
echo " Credentials: $CREDS_FILE"
echo " Auth token: ${AUTH_TOKEN:0:30}..."
echo ""
echo "Now use /uat to test the agent:"
echo " /uat What's the ticker for Tesla?"
echo " /uat Show me my portfolio summary"
name description argument-hint disable-model-invocation allowed-tools
uat-verify
Visually verify Ghostfolio UI state using agent-browser after agent actions. Use after /uat add_activity calls to confirm the UI reflects the change.
activities|portfolio|holdings
true
Bash, Read

UAT-Verify - Visual UI Verification

Navigate to the Ghostfolio UI, log in as the test user, and screenshot the $ARGUMENTS page. Then analyze the screenshot to confirm expected state.

Valid views: activities, portfolio, holdings

Prerequisites

  • .uat-creds.json must exist (run /uat first or bash .claude/skills/uat/scripts/setup.sh)
  • Ghostfolio UI must be running (same dev server as API, typically at http://localhost:4200 for local dev or http://localhost:3333 for Docker)
  • agent-browser must be installed globally

Steps

  1. Read credentials from .uat-creds.json:

    { "accessToken": "...", "authToken": "..." }
  2. Determine URLs based on the $ARGUMENTS view:

    • activities -> /transactions page
    • portfolio -> /home page (portfolio summary is on the dashboard)
    • holdings -> /home page (scroll down for holdings breakdown)

    Base URL: try http://localhost:4200 first (Angular dev server). Fall back to http://localhost:3333 (Docker full stack).

  3. Check UI is reachable:

    curl -sf http://localhost:4200 > /dev/null 2>&1 && echo "4200" || echo "3333"
  4. Log in via agent-browser using the anonymous access token:

    • Navigate to base URL
    • The anonymous login flow: go to /auth/anonymous?accessToken=<accessToken> directly, which auto-logs in
    • Wait for redirect to complete (wait for dashboard element)
  5. Navigate to the target page and take a screenshot:

    For activities:

    agent-browser open "http://localhost:4200/auth/anonymous?accessToken=<accessToken>" && \
    agent-browser wait ".mat-mdc-row, .no-transactions, [data-testid='transactions']" && \
    agent-browser open "http://localhost:4200/transactions" && \
    agent-browser wait ".mat-mdc-row, .no-transactions" && \
    agent-browser screenshot screenshots/uat-activities.png

    For portfolio:

    agent-browser open "http://localhost:4200/auth/anonymous?accessToken=<accessToken>" && \
    agent-browser wait "gf-portfolio-summary, .portfolio-summary, main" && \
    agent-browser screenshot screenshots/uat-portfolio.png

    For holdings:

    agent-browser open "http://localhost:4200/auth/anonymous?accessToken=<accessToken>" && \
    agent-browser wait "gf-holdings, .holdings, main" && \
    agent-browser screenshot screenshots/uat-holdings.png
  6. Take accessibility snapshot for structured analysis:

    agent-browser snapshot
  7. Analyze the screenshot and snapshot - Report:

    • What's visible on the page (holdings, activities, balances)
    • Whether the expected data matches (e.g., after /uat Buy 10 AAPL at 180, confirm "AAPL BUY 10" appears)
    • Any error states or empty states

Troubleshooting

  • agent-browser not found: Install globally with npm install -g agent-browser
  • UI not reachable on 4200: Check if npm run start:client is running. If using Docker only, try 3333.
  • Login redirect fails: Try navigating directly to the target page after the auth URL.
  • Screenshot blank/loading: Add agent-browser wait 2000 before screenshot for slower pages.
  • Wrong port: Check $FORWARDED_PORTS env var if in a devcontainer.

Screenshot output

Screenshots are saved to screenshots/ (gitignored):

  • screenshots/uat-activities.png
  • screenshots/uat-portfolio.png
  • screenshots/uat-holdings.png
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment