Chat Orchestration
How AI conversations work in VirtuousAI - providers, context, and tool execution
Chat Orchestration
VirtuousAI's chat system is more than a simple chatbot. It's an orchestrator that understands context, executes actions, and maintains conversation state across sessions.
How It Works
When you send a message, here's what happens behind the scenes:
Key Components
The Orchestrator
The orchestrator is the "brain" of the chat system. It:
- Understands context — Knows who you are, your timezone, and your organization
- Chooses actions — Decides whether to answer directly or use tools
- Maintains focus — Keeps conversations on track across multiple turns
- Streams responses — Delivers answers progressively for a responsive feel
Context Injection
Every conversation includes rich context:
| Context | What It Provides |
|---|---|
| User | Your name, email, and preferences |
| Organization | Your team and permissions |
| Temporal | Current date/time in your timezone |
| Conversation | Recent message history |
This context helps the assistant personalize responses and understand time-relative questions like "Show me yesterday's sales."
Message Flow
Messages flow through a structured pipeline:
- User sends message → Stored with thread ID
- Turn created → Tracks this request/response cycle
- Context built — User, org, and temporal context assembled
- Agent processes → Orchestrator analyzes and responds
- Response streamed → Answer delivered progressively
- Turn completed → Results stored for history
Providers and Models
VirtuousAI uses multiple AI providers for resilience. If one provider is unavailable, the system automatically falls back to alternatives.
You don't choose models directly — we select the best model for each task. This keeps things simple and ensures consistent quality.
Provider Fallback
The system tries providers in order until one succeeds:
- Primary provider — Handles most requests
- Secondary provider — Takes over if primary fails
- Tertiary provider — Last resort fallback
This multi-provider approach ensures high availability even during provider outages.
Tool Execution
When the assistant needs to take action (not just answer), it uses tools. Tools are discrete operations that:
- Execute within time budgets
- Create audit trails via action runs
- Return structured results
Available Tools
| Tool | Category | Description |
|---|---|---|
generate_query | Data | Generate and execute SQL queries against connected schemas |
explain_schema | Data | Explain table structures and column meanings |
web_search | Research | Search the web for information via Tavily API |
fetch_page | Research | Extract readable content from web page URLs |
call_agent | Delegation | Invoke a specialized agent by slug |
create_flashboard | Output | Build a persistent flashboard dashboard |
http_request | Integration | Make arbitrary HTTP requests to external APIs |
Tool Safety
Tools are categorized by risk level:
| Level | Description | Example |
|---|---|---|
| Read | Only retrieves data | generate_query, web_search, fetch_page |
| Write | Modifies data or creates resources | create_flashboard, http_request |
Write tools may require explicit approval before execution.
Agent Switching
Chat supports switching between different AI agents mid-conversation. Each agent has specialized tools and instructions for its domain.
- Primary agents appear in the agent picker and can be selected by the user
- Subagents execute behind the scenes when a primary agent delegates via
call_agent - The orchestrator routes messages to the active agent, which executes with its own toolset
See AI Agents for details on how agents compose and delegate.
Crystallization
Successful chat threads with completed actions can be crystallized into deterministic automations. This captures the sequence of tool calls as a reusable workflow.
See Crystallization for details.
Conversation State
Conversations are organized into threads and turns:
- Thread — A conversation topic (persists across sessions)
- Turn — A single request/response exchange
- Message — User input or assistant output
This structure enables:
- Resuming conversations later
- Reviewing conversation history
- Auditing what actions were taken
Settings and Limits
The chat system has built-in guardrails:
| Setting | Purpose | Default |
|---|---|---|
| Turn timeout | Max time per response | 9 minutes |
| Tool call limit | Max tools per turn | 50 |
| Context window | How much history to include | ~80% of model limit |
These limits prevent runaway operations and ensure responsive interactions.
Best Practices
Be Specific
Instead of "Show me data," try "Show me orders from last week by product category."
Ask Follow-ups
The assistant remembers context. After asking about sales, you can say "Break that down by region" without repeating yourself.
Check Tool Results
When the assistant executes actions, it shows what it did. Review these to understand changes made on your behalf.