Atmosphere AI Framework Abstraction — Implementation Summary

Overview

Atmosphere's unified AI abstraction layer sits between @AiEndpoint handlers and the four supported AI frameworks (Spring AI, LangChain4j, Google ADK, Embabel). It provides:

• Tool Calling SPI — @AiTool annotation → framework-agnostic ToolDefinition → native bridges
• Capability Discovery — AiCapability enum + ModelRouter for smart routing/failover
• Conversation Memory — ConversationPersistence SPI backed by Redis or SQLite
• Guardrails — sealed GuardrailResult (Pass/Modify/Block) pipeline
• RAG — ContextProvider SPI for retrieval-augmented generation
• Retry/Circuit Breaker — RetryPolicy record + DefaultModelRouter health tracking
• Multi-Modal Content — Content sealed interface (Text/Image/File)
• Observability — AiMetrics SPI aligned with OpenTelemetry GenAI conventions

Architecture

@AiEndpoint handler
    │
    ▼
AiStreamingSession.stream(message)
    │
    ├── ToolRegistry.allTools() → AiRequest.withTools(...)
    ├── AiGuardrail.inspectRequest() → Pass / Modify / Block
    ├── ContextProvider.retrieve() → RAG augmentation
    ├── AiInterceptor.preIntercept()
    │
    ▼
AiSupport.stream(request, session)      ← adapter selected by ModelRouter
    │
    ├── SpringAiSupport   → SpringAiToolBridge → ToolCallback[]
    ├── LangChain4jSupport → LangChain4jToolBridge → ToolSpecification[]
    ├── AdkAiSupport      → AdkToolBridge → BaseTool[]
    └── EmbabelAiSupport  → (agent orchestration)
    │
    ▼
StreamingSession → WebSocket/SSE/gRPC client

Pipeline Order

Tools → Guardrails (pre) → Context Providers (RAG) → Interceptors (pre)
    → [LLM call] →
Interceptors (post) → Client

New Files Created

Core SPI (modules/ai)

File	Purpose
AiCapability.java	9-value enum for feature discovery
Content.java	Sealed interface: Text, Image, File records
AiGuardrail.java	Pre/post guardrails with sealed GuardrailResult
ContextProvider.java	RAG SPI with Document record
ModelRouter.java	Routing SPI with FallbackStrategy enum
DefaultModelRouter.java	Circuit breaker + FAILOVER/ROUND_ROBIN/CONTENT_BASED
RetryPolicy.java	Exponential backoff with jitter
AiMetrics.java	OpenTelemetry-aligned observability SPI
ConversationPersistence.java	Thin persistence SPI (load/save/remove)
PersistentConversationMemory.java	Write-through cache + sliding window
annotation/AiTool.java	Method annotation for AI-callable tools
annotation/Param.java	Parameter annotation for tool methods
tool/ToolDefinition.java	Framework-agnostic tool record + builder
tool/ToolParameter.java	Parameter metadata with JSON Schema types
tool/ToolExecutor.java	Functional interface for tool execution
tool/ToolResult.java	Execution result record (success/failure)
tool/ToolRegistry.java	Tool registry interface
tool/DefaultToolRegistry.java	ConcurrentHashMap-backed + @AiTool scanning

Tool Bridges (adapter modules)

File	Framework	How Tools Are Registered
SpringAiToolBridge.java	Spring AI 2.0	ToolDefinition → ToolCallback → promptSpec.toolCallbacks(...)
LangChain4jToolBridge.java	LangChain4j 1.0	ToolDefinition → ToolSpecification → ChatRequest.builder().toolSpecifications(...)
ToolAwareStreamingResponseHandler.java	LangChain4j 1.0	Handles tool call loop (execute → re-submit, max 5 rounds)
AdkToolBridge.java	Google ADK 0.2	ToolDefinition → BaseTool subclass → LlmAgent.builder().tools(...)

Persistence Backends

File	Backend	Key Design
RedisConversationPersistence.java	Redis (Lettuce)	Key prefix atmosphere:conversation:, SETEX with TTL
SqliteConversationPersistence.java	SQLite (embedded)	Table ai_conversations, WAL mode, in-memory option

Modified Files

• AiStreamingAdapter.java — added default capabilities()
• AiSupport.java — added default capabilities()
• StreamingSession.java — added default sendContent(Content)
• AiRequest.java — added withTools() / tools() for tool transport
• AiEndpoint.java — added tools[], excludeTools[], guardrails[], contextProviders[], fallbackStrategy
• AiStreamingSession.java — full pipeline: tools → guardrails → RAG → interceptors → LLM
• AiEndpointProcessor.java — tool scanning, guardrail/context instantiation
• AiEndpointHandler.java — passes tools/guardrails/context to session
• All adapter *Support.java files — added capabilities() and tool bridge wiring

Tool Calling Flow — How @AiTool Reaches the LLM

// 1. Developer declares a tool
public class WeatherTools {
    @AiTool(name = "get_weather", description = "Get weather for a city")
    public String getWeather(@Param("city") String city) {
        return weatherService.get(city);
    }
}

// 2. @AiEndpoint selects tools
@AiEndpoint(path = "/chat", tools = WeatherTools.class)

// 3. AiEndpointProcessor scans and registers
registry.register(new WeatherTools());  // reflection-based

// 4. AiStreamingSession.stream() attaches tools to request
request = request.withTools(toolRegistry.allTools());

// 5. Adapter bridges to native format
// Spring AI:   SpringAiToolBridge.toToolCallbacks(tools) → promptSpec.toolCallbacks(...)
// LangChain4j: LangChain4jToolBridge.toToolSpecifications(tools) → chatRequest.toolSpecifications(...)
// ADK:         AdkToolBridge.toAdkTools(tools) → LlmAgent.builder().tools(...)

// 6. Framework handles tool call loop automatically (Spring AI, ADK)
//    or ToolAwareStreamingResponseHandler manages it (LangChain4j)

Test Coverage

75 tests across 4 modules, all passing:

Module	Test Class	Tests
atmosphere-ai	DefaultToolRegistryTest	11 — register, scan, execute, annotated tools
atmosphere-ai	ToolDefinitionTest	10 — builder, validation, JSON schema types, ToolResult
atmosphere-ai	ContentTest	9 — sealed variants, base64, validation, pattern matching
atmosphere-ai	RetryPolicyTest	8 — backoff, jitter, cap, shouldRetry
atmosphere-ai	PersistentConversationMemoryTest	10 — CRUD, persistence across instances, eviction
atmosphere-ai	DefaultModelRouterTest	10 — failover, round-robin, capability filter, health
atmosphere-durable-sessions-sqlite	SqliteConversationPersistenceTest	8 — CRUD, large payloads, special chars
atmosphere-spring-ai	SpringAiToolBridgeTest	9 — schema gen, arg parsing, callback execution
atmosphere-langchain4j	LangChain4jToolBridgeTest	8 — spec conversion, tool execution, arg parsing

Design Decisions

1. Hybrid @AiTool scope — tools registered globally in ToolRegistry, selected per-endpoint via @AiEndpoint(tools=...) and excludeTools=...

2. ConversationPersistence as thin SPI — only 3 methods (load/save/remove). Redis and SQLite implementations in their respective durable-sessions modules with optional dependency on atmosphere-ai.

3. Manual JSON serialization in PersistentConversationMemory to avoid adding Jackson dependency to the core AI module.

4. LangChain4j tool loop handled by ToolAwareStreamingResponseHandler (max 5 rounds) since LangChain4j doesn't auto-execute tool callbacks in streaming mode.

5. ADK tools at build time — ADK requires tools at agent construction. AdkAiSupport.configureWithTools() method provided for this. Runtime request tools logged with guidance.

6. Circuit breaker in DefaultModelRouter — consecutive failure tracking, cooldown-based recovery, last-resort fallback to all backends when all healthy ones exhausted.