> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vibesflow.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Alith Orchestrator

> AI-powered music generation orchestrator using Alith framework

# Alith Orchestrator

The Alith Orchestrator is an intelligent music generation system that combines the [Alith AI framework](https://lazai.network/alith) with Google's Lyria RealTime API to provide zero-dropout audio streaming and adaptive music generation.

## Architecture Overview

The orchestrator implements a distributed architecture with multiple specialized components:

```mermaid theme={null}
graph TB
    subgraph "Backend Orchestrator"
        AO[AlithOrchestrator] --> AA[AlithAgent<br/>Gemini 2.5 Flash Lite]
        AO --> ABM[AlithBufferManager]
        AO --> SI[SensorInterpreter]
        AO --> WS[WebSocket Server]
    end
    
    subgraph "Frontend Coordinator"
        OC[OrchestrationCoordinator] --> LS[LyriaSession]
        OC --> GBM[GeminiBufferManager]
        OC --> SC[ServerConnection]
    end
    
    subgraph "External Services"
        LYRIA[Lyria RealTime API<br/>Google Music Generation]
        GEMINI[Gemini API<br/>Predictive Analysis]
        QDRANT[QDrant Store<br/>Pattern Learning]
    end
    
    AA --> GEMINI
    ABM --> QDRANT
    LS --> LYRIA
    GBM --> GEMINI
    SC --> WS
    
    classDef alith fill:#e1f5fe
    classDef google fill:#fff3e0
    classDef storage fill:#f3e5f5
    
    class AO,AA,ABM alith
    class LYRIA,GEMINI google
    class QDRANT storage
```

## Core Components

### AlithAgent

The core intelligence uses Alith's Agent class with Gemini 2.5 Flash Lite for real-time decision making:

```javascript theme={null}
// Initialize Alith agent with optimized preamble
musicAgent = new Agent({
  model: "gemini-2.5-flash-lite",
  apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
  baseUrl: "generativelanguage.googleapis.com/v1beta/openai",
  preamble: enhancedPreamble,
  memory: new WindowBufferMemory(8)
});
```

The agent processes sensor data and generates Lyria prompts using embedded knowledge:

```javascript theme={null}
const interpretation = await musicAgent.run(JSON.stringify({
  sensorData: enrichedSensorData,
  sessionHistory: sessionData?.history || [],
  userProfile: sessionData?.profile || {},
  currentBaseline: this.currentBaseline
}));
```

### Enhanced Sensor Processing

The orchestrator implements intelligent sensor interpretation with rate limiting and smoothing:

```mermaid theme={null}
sequenceDiagram
    participant App as App/Sensors
    participant Coord as Coordinator
    participant Server as Orchestrator
    participant Agent as AlithAgent
    participant Lyria as Lyria API
    
    App->>Coord: Sensor data (accelerometer/gyro)
    Coord->>Coord: Rate limiting (250ms)
    Coord->>Server: WebSocket sensor data
    Server->>Agent: Process with context
    Agent->>Agent: Generate interpretation
    Agent->>Server: Lyria prompt + config
    Server->>Lyria: Stream music generation
    Lyria->>Coord: Audio chunks (16-bit PCM)
    Coord->>App: Buffer and play audio
```

### Rate Limiting and Optimization

The system implements sophisticated rate limiting to respect API quotas:

```javascript theme={null}
// Optimized rate limiting for baseline-driven processing
this.geminiCallCooldown = 4500; // 4.5 seconds for 15 requests/minute quota
this.serverLatency = 250; // Stability-focused latency
this.minSendInterval = 200; // Smooth server processing
```

## Pattern Learning System

### QDrant Integration

User patterns are stored using QDrant vector database for continuous learning:

```javascript theme={null}
userPatternStore = new QdrantStore({
  url: process.env.QDRANT_URL || 'http://localhost:6333',
  collectionName: 'user_patterns',
  apiKey: process.env.QDRANT_API_KEY
});
```

### Memory Management

The system uses Alith's WindowBufferMemory for context retention:

```javascript theme={null}
// Reduced memory window for token efficiency
memory: new WindowBufferMemory(8)
```

## WebSocket Communication

### Server Implementation

The orchestrator provides WebSocket endpoints for real-time communication:

```javascript theme={null}
this.wsServer.on('connection', (ws) => {
  ws.on('message', async (message) => {
    const data = JSON.parse(message);
    
    if (data.type === 'sensorUpdate') {
      await this.processSensorData(data.sensorData, ws);
    }
  });
});
```

### Client Coordination

The frontend coordinator manages multiple connection types:

```mermaid theme={null}
graph LR
    subgraph "OrchestrationCoordinator"
        SC[ServerConnection<br/>Enhanced Interpretation]
        LS[LyriaSession<br/>Direct Music Generation]
        GBM[GeminiBufferManager<br/>Predictive Buffering]
    end
    
    SC --> |WebSocket| Backend[Backend Orchestrator]
    LS --> |WebSocket| Lyria[Lyria RealTime]
    GBM --> |HTTP| Gemini[Gemini API]
    
    SC -.-> |Fallback| LS
    LS --> |Audio Chunks| GBM
```

## Performance Optimizations

### Token Efficiency

The system optimizes token usage through embedded knowledge:

```javascript theme={null}
function preambleWithKnowledge() {
  const poemsContent = readFileSync(join(__dirname, 'knowledge', 'poems.txt'), 'utf8');
  const parametersContent = readFileSync(join(__dirname, 'knowledge', 'parameters.txt'), 'utf8');
  
  return `Enhanced preamble with ${poemsContent} and ${parametersContent}`;
}
```

### Processing Queue

Agent synchronization prevents parallel token consumption:

```javascript theme={null}
let agentProcessingQueue = Promise.resolve();
let activeProcessingCount = 0;

// Queue processing to prevent parallel API calls
agentProcessingQueue = agentProcessingQueue.then(async () => {
  activeProcessingCount++;
  try {
    return await processRequest();
  } finally {
    activeProcessingCount--;
  }
});
```

## Error Handling and Resilience

### Fallback Systems

The orchestrator implements multiple fallback mechanisms:

```javascript theme={null}
createIntelligentRaveFallback(enrichedSensorData, sessionData) {
  // Energy-based fallback when AI agent is unavailable
  const energy = Math.sqrt(
    Math.pow(enrichedSensorData.x, 2) + 
    Math.pow(enrichedSensorData.y, 2) + 
    Math.pow(enrichedSensorData.z, 2)
  ) / 3;
  
  return {
    singleCoherentPrompt: this.generateEnergyBasedPrompt(energy),
    lyriaConfig: this.generateEnergyBasedConfig(energy),
    requiresCrossfade: energy > 0.6
  };
}
```

### Connection Recovery

Automatic reconnection with exponential backoff:

```javascript theme={null}
async reconnectToServer() {
  const delay = Math.min(1000 * Math.pow(2, this.reconnectAttempts), 30000);
  
  setTimeout(async () => {
    try {
      await this.connectToInterpretationServer();
    } catch (error) {
      this.reconnectAttempts++;
      if (this.reconnectAttempts < this.maxReconnectAttempts) {
        this.reconnectToServer();
      }
    }
  }, delay);
}
```