RFP-Solver Pipeline Architecture

A detailed breakdown of the multi-agent pipeline for answering RFP security questions using Wiz documentation.

Pipeline Overview

flowchart TB
    subgraph Input
        Q[/"RFP Question"/]
    end

    subgraph "Pre-Pipeline (Sequential)"
        C[🏷️ Classifier Agent]
        QP[📋 Query Planner Agent]
    end

    subgraph "Answer Pipeline (SequentialAgent)"
        R[🔍 Research Agent]
        E[📝 Evidence Agent]
        
        subgraph "Synthesis-Critic Loop (LoopAgent)"
            S[✍️ Synthesis Agent]
            CR[🔎 Critic Agent]
        end
    end

    subgraph Output
        RESULT[/"Pipeline Result"/]
    end

    Q --> C
    C --> QP
    QP --> R
    R --> E
    E --> S
    S --> CR
    CR -->|"REVISE"| S
    CR -->|"PASS"| RESULT

    style C fill:#e1f5fe
    style QP fill:#e1f5fe
    style R fill:#fff3e0
    style E fill:#fff3e0
    style S fill:#e8f5e9
    style CR fill:#fce4ec

Loading

Detailed Agent Flow with Data Structures

sequenceDiagram
    participant User
    participant Runner
    participant Classifier
    participant QueryPlanner
    participant Research
    participant Evidence
    participant Synthesis
    participant Critic

    User->>Runner: question: string
    
    Note over Runner: Initialize session state
    
    Runner->>Classifier: question
    Classifier-->>Runner: ClassificationResult
    
    Runner->>QueryPlanner: question + ClassificationResult
    QueryPlanner-->>Runner: QueryPlanResult
    
    Note over Runner: Create AnswerPipeline session<br/>with initial state
    
    Runner->>Research: state[question, query_plan]
    Research-->>Runner: ResearchResult → state[research]
    
    Runner->>Evidence: state[question, query_plan, research]
    Evidence-->>Runner: EvidenceResult → state[evidence]
    
    loop Synthesis-Critic Loop (max 3 iterations)
        Runner->>Synthesis: state[question, evidence, critic?]
        Synthesis-->>Runner: SynthesisResult → state[synthesis]
        
        Runner->>Critic: state[question, synthesis, evidence]
        Critic-->>Runner: CriticResult → state[critic]
        
        alt verdict == PASS
            Critic->>Runner: approve_answer() → escalate=true
            Note over Runner: Exit loop
        else verdict == REVISE
            Note over Runner: Continue loop with<br/>revision_instructions
        end
    end
    
    Runner-->>User: PipelineResult

Loading

---

Agent Details

1. Classifier Agent

Purpose: Determines if the question requires CSP-specific answers (AWS/GCP/Azure) or a single cloud-agnostic answer.

Input:

// Raw question string
"Does Wiz support AWS GuardDuty integration?"

Output:

interface ClassificationResult {
  is_csp_specific: boolean;      // true if needs per-CSP answers
  reasoning: string;             // "Question mentions AWS GuardDuty..."
  aws_question?: string | null;  // CSP-specific variant
  gcp_question?: string | null;
  azure_question?: string | null;
}

Tools: None (pure LLM reasoning)

---

2. Query Planner Agent

Purpose: Decomposes the question into searchable facets with Wiz-specific terminology.

Input:

// Question + Classification context
`Question: Does Wiz support AWS GuardDuty integration?

Classification: CSP-specific
Reasoning: Question mentions AWS GuardDuty...`

Output:

interface QueryPlanResult {
  original_question: string;
  facets: QueryFacet[];           // Decomposed aspects of the question
  primary_queries: string[];      // Top 3-5 search queries
  reasoning: string;              // Why these facets/queries
}

interface QueryFacet {
  name: string;                   // "GuardDuty Integration"
  description: string;            // "How Wiz integrates with AWS GuardDuty"
  wiz_terms: string[];            // ["Cloud Detection", "Threat Detection"]
  search_queries: string[];       // ["Wiz GuardDuty integration", ...]
  target_docs?: string[];         // Optional doc paths
}

Tools: None (pure LLM reasoning with Wiz glossary context)

---

3. Research Agent

Purpose: Searches Wiz documentation to find relevant sources for each facet.

Input (from session state):

state['question']    // Original RFP question
state['query_plan']  // JSON string of QueryPlanResult

Output:

interface ResearchResult {
  sources: ResearchSource[];      // 2-6 sources depending on complexity
  confidence: number;             // 0-100 confidence score
  capability_gap?: string | null; // If Wiz doesn't support this
}

interface ResearchSource {
  title: string;        // "Cloud Detection and Response"
  url: string;          // "https://docs.wiz.io/docs/cloud-detection"
  source_path: string;  // "packages/docs-web/content/guides/cdr.mdx"
  content: string;      // Snippet from search results
  csp?: string;         // "AWS" | "GCP" | "Azure" | "General"
}

Tools:

• query_docs(query, csp?, max_results?) - Search via Gemini File Search API
• read_document(path) - Read full document content

---

4. Evidence Agent

Purpose: Extracts verbatim quotes from sources to ground the answer.

Input (from session state):

state['question']    // Original RFP question
state['query_plan']  // Query plan for context
state['research']    // JSON string of ResearchResult

Output:

interface EvidenceResult {
  evidence: SourceEvidence[];
}

interface SourceEvidence {
  source_path: string;   // Path to document
  source_title: string;  // Document title
  quotes: string[];      // Verbatim quotes from document
}

Tools:

• read_document(path) - Read full document for quote extraction

---

5. Synthesis Agent

Purpose: Writes the final answer using ONLY the extracted quotes.

Input (from session state):

state['question']   // Original RFP question
state['evidence']   // JSON string of EvidenceResult
state['critic']     // Previous critic feedback (if revision)

Output:

interface SynthesisResult {
  answer: string;                    // The synthesized answer
  compliance_status: ComplianceStatus; // "Fully Supported" | "Partially Supported" | "Not Supported"
  confidence_score: number;          // 0-100
  citations: Citation[];             // Sources for each claim
  facets_covered: string[];          // Which facets were addressed
  facets_missing: string[];          // Which facets lack evidence
  reasoning: string;                 // Why this compliance/confidence
}

interface Citation {
  source_path: string;
  source_title: string;
  url: string;
  quote: string;        // Verbatim quote used
}

Tools: None (uses outputSchema for structured output)

---

6. Critic Agent

Purpose: Validates the synthesized answer against quality criteria.

Input (from session state):

state['question']   // Original RFP question
state['synthesis']  // JSON string of SynthesisResult
state['evidence']   // JSON string of EvidenceResult

Output:

interface CriticResult {
  verdict: CriticVerdict;                    // "PASS" | "REVISE" | "FAIL"
  checks: Record<string, ValidationCheck>;   // Individual check results
  issues?: string[];                         // Problems found
  suggestions?: string[];                    // Improvement suggestions
  recommended_confidence?: number;           // Adjusted confidence
  revision_instructions?: string | null;     // How to fix (if REVISE)
}

interface ValidationCheck {
  passed: boolean;
  issue?: string | null;
}

// Validation checks performed:
// - quote_grounding: All claims backed by quotes?
// - status_alignment: Compliance status matches evidence?
// - confidence_calibration: Score appropriate for evidence?
// - facet_coverage: All facets addressed?
// - answer_quality: Clear, professional, complete?

Tools:

• approve_answer() - Signals loop exit when verdict is PASS

---

Session State Flow

stateDiagram-v2
    [*] --> Init: Runner creates session

    Init --> Classified: Classifier writes
    state Init {
        question: string
        csp: string
    }

    Classified --> Planned: QueryPlanner writes
    state Classified {
        classification: ClassificationResult
    }

    Planned --> Researched: Research writes
    state Planned {
        query_plan: QueryPlanResult
    }

    Researched --> Evidenced: Evidence writes
    state Researched {
        research: ResearchResult
    }

    Evidenced --> Synthesized: Synthesis writes
    state Evidenced {
        evidence: EvidenceResult
    }

    Synthesized --> Critiqued: Critic writes
    state Synthesized {
        synthesis: SynthesisResult
    }

    Critiqued --> Synthesized: REVISE
    Critiqued --> [*]: PASS
    state Critiqued {
        critic: CriticResult
    }

Loading

---

State Key Reference

Key	Written By	Type	Description
question	Runner (init)	string	Original RFP question
csp	Runner (init)	string	CSP context from classifier
classification	Classifier	ClassificationResult	CSP-specific determination
query_plan	Query Planner	QueryPlanResult	Decomposed facets and queries
research	Research Agent	ResearchResult	Found sources with content
evidence	Evidence Agent	EvidenceResult	Extracted quotes per source
synthesis	Synthesis Agent	SynthesisResult	Final answer with citations
critic	Critic Agent	CriticResult	Validation result and feedback

---

Final Pipeline Result

interface PipelineResult {
  question: string;                    // Original question
  classification: ClassificationResult; // CSP classification
  queryPlan: QueryPlanResult;          // Search strategy
  research: ResearchResult;            // Found sources
  evidence: EvidenceResult;            // Extracted quotes
  synthesis: SynthesisResult;          // Final answer
  synthesisHistory: SynthesisResult[]; // All synthesis attempts
  criticFeedback: CriticResult[];      // All critic feedback
  iterations: number;                  // How many synthesis-critic loops
  duration: number;                    // Total time in ms
  success: boolean;                    // Pipeline completed successfully
  error?: string;                      // Error message if failed
}

---

Architecture Notes

1. ADK Framework: Built on Google's Agent Development Kit (ADK) using LlmAgent, SequentialAgent, and LoopAgent primitives.

2. State Management: Agents communicate via session.state using outputKey to write results.

3. Tool Constraints: ADK doesn't allow both tools and outputSchema on the same agent:

   • Agents with tools (Research, Evidence, Critic) output JSON via prompt instructions
   • Agents without tools (Classifier, QueryPlanner, Synthesis) use outputSchema

4. Parallel Execution: Agent factories (createXAgent()) return fresh instances to support parallel pipeline execution.

5. Loop Exit: The Critic calls approve_answer() tool which sets escalate=true to exit the LoopAgent.