Skip to content

Instantly share code, notes, and snippets.

@schpet

schpet/allspark-model-usage-analysis.md

Created January 15, 2026 21:42
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save schpet/102b9978398fb275236c75a5d50a39b9 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save schpet/102b9978398fb275236c75a5d50a39b9 to your computer and use it in GitHub Desktop.
Download ZIP
ALL-43: AllSpark Builder AI Model Usage Analysis
Raw
allspark-model-usage-analysis.md

AllSpark Builder - AI Model Usage Analysis

Issue: ALL-43 - Allow user to choose model (opus or sonnet) in the builder UI

Date: 2026-01-15

Overview

AllSpark Builder uses multiple AI models across different services. This document maps out which services use which models, whether they have user-configurable model selection, and identifies where model selection should be added.

Model Usage by Category

1. Build Sessions (Claude Code SDK)

These services power the main "Builder UI" where users build their target applications.

Service Location Default Model Configurable?
ClaudeCodeRubyService app/services/claude_code_ruby_service.rb:765 CLAUDE_CODE_MODEL env → claude-opus-4-1-20250805 NO - env var only
ClaudeCodeApiService app/services/claude_code_api_service.rb:104 CLAUDE_MODEL env → claude-3-5-sonnet-20241022 NO - env var only

Key Finding: The primary build session service defaults to Opus but has no UI for users to select between Opus and Sonnet.

2. Sprint Zero / Planning Services

These generate artifacts (PRDs, personas, user stories, etc.) for the target application being built.

Service Location Default Provider Default Model Configurable?
Llm::PrdGeneratorService app/services/llm/prd_generator_service.rb:29 gemini gemini-2.5-flash NO
Ai::PersonaGenerator app/services/ai/persona_generator.rb:22 openrouter google/gemini-3-pro-preview NO
Ai::StoryGenerator app/services/ai/story_generator.rb:36 gemini gemini-2.5-flash NO
Ai::BusinessCaseGenerator app/services/ai/business_case_generator.rb:20 gemini gemini-2.5-pro NO
Ai::JourneyFlowGenerator app/services/ai/journey_flow_generator.rb:11 openai gpt-4o-mini NO
Ai::JourneyDiscoveryService app/services/ai/journey_discovery_service.rb:11 openai gpt-4o-mini NO
Ai::SitemapGeneratorV3 app/services/ai/sitemap_generator_v3.rb:16 openai (provider default) NO
Ai::DomainModelGenerator app/services/ai/domain_model_generator.rb:11 openai (provider default) NO
Ai::DomainModelAssistant app/services/ai/domain_model_assistant.rb:21 gemini (provider default) NO
Ai::ScreenDesigner app/services/ai/screen_designer.rb:78 gemini (provider default) NO

3. Screen Design Services

Service Location Default Model Configurable?
ScreenDesign::GeneratorService app/services/screen_design/generator_service.rb:248 claude-3-5-sonnet-20241022 NO
ScreenDesign::TemplateCustomizer app/services/screen_design/template_customizer.rb:177 claude-sonnet-4-5-20250929 NO

4. Other LLM Services

Service Location Default Provider/Model Configurable?
Llm::ClaudeMdGeneratorService app/services/llm/claude_md_generator_service.rb:314 claude or openai NO
Llm::MarketingPageGeneratorService app/services/llm/marketing_page_generator_service.rb:12 gemini NO
Llm::LogoGeneratorService app/services/llm/logo_generator_service.rb:21 openai NO
SprintZero::LlmEnrichmentService app/services/sprint_zero/phase_generators/llm_enrichment_service.rb:111 gpt-4-turbo-preview NO

5. Assistant Model (Target App AI Features)

The Assistant model is the only service with built-in model selection.

Service Location Configurable? Selection Method
Assistant app/models/assistant.rb:211-314 YES llm_model_name + model_provider attributes

Capabilities:

  • llm_model_name attribute - stores the model name (e.g., "claude-3-5-sonnet-20241022")
  • model_provider attribute - explicit provider selection (openrouter, google, anthropic, openai)
  • Auto-detects provider from model name if not explicitly set
  • Supports: OpenRouter, Google Gemini, Anthropic Claude, OpenAI

Central Configuration

Model defaults are centralized in app/services/llm/configuration.rb:

# Default models (can be overridden via ENV)
openrouter_model: ENV.fetch("OPENROUTER_MODEL", "google/gemini-3-pro-preview")
openai_model: ENV.fetch("OPENAI_MODEL", "gpt-5")
claude_model: ENV.fetch("CLAUDE_MODEL", "claude-sonnet-4-5-20250929")
gemini_model: ENV.fetch("GEMINI_MODEL", "gemini-2.5-flash")

Environment Variables

Variable Purpose Default
LLM_PROVIDER Primary provider for most services openai
CLAUDE_CODE_MODEL Build sessions (Claude Code SDK) claude-opus-4-1-20250805
CLAUDE_MODEL Claude API calls claude-sonnet-4-5-20250929
OPENAI_MODEL OpenAI API calls gpt-5
GEMINI_MODEL Gemini API calls gemini-2.5-flash
OPENROUTER_MODEL OpenRouter API calls google/gemini-3-pro-preview

Recommendations for ALL-43

Primary Target: Build Sessions

The build session is the main "builder UI" where users interact with Claude to build their applications. Currently:

  1. ClaudeCodeRubyService uses CLAUDE_CODE_MODEL env var (defaults to Opus)
  2. No UI exists to let users choose between models
  3. Model selection could be added at:
    • BuildSession level - per-session model choice
    • AppProject level - default model for all sessions in a project
    • User level - user's preferred default model

Suggested Implementation Locations

  1. Add model field to BuildSession or ClaudeCodeSession model
  2. Add UI selector in build session creation/settings
  3. Pass model to ClaudeCodeRubyService via options

Model Options to Offer

For Claude models via the Claude Code SDK:

  • claude-opus-4-1-20250805 (current default - most capable, higher cost)
  • claude-sonnet-4-5-20250929 (faster, lower cost)

Architecture Summary

┌─────────────────────────────────────────────────────────────┐
│                     BUILDER APPLICATION                      │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌─────────────────────────────────────────────────────┐    │
│  │         BUILD SESSIONS (Claude Code SDK)            │    │
│  │  ClaudeCodeRubyService → CLAUDE_CODE_MODEL (opus)   │◄───┼── NEEDS MODEL SELECTOR
│  │  ClaudeCodeApiService → CLAUDE_MODEL (sonnet)       │    │
│  └─────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌─────────────────────────────────────────────────────┐    │
│  │              SPRINT ZERO SERVICES                    │    │
│  │  PrdGenerator, PersonaGenerator, StoryGenerator...  │    │
│  │  (Various providers: gemini, openai, openrouter)    │    │
│  └─────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌─────────────────────────────────────────────────────┐    │
│  │              ASSISTANT MODEL                         │    │
│  │  Has llm_model_name + model_provider attributes     │◄───┼── ALREADY HAS SELECTOR
│  └─────────────────────────────────────────────────────┘    │
│                                                              │
└─────────────────────────────────────────────────────────────┘
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment